If I had a virus or spy ware binary, not the source code, and I wanted to determine the purpose, functionality, basically as much as I could find out about that software by monitoring it…what tools could I use and methods? I'm quite interested in this application to Linux and Windows testing/tools only.
This is theory, not real case.
You could write a whole book about this topic and, in fact, people have.
Consider
Malware Forensics, Aquilina, Casey and Malin
http//
Malware Fighting Malicious Code, Skoudis and Zeltzer
http//
For starters…
Thanks for that, I just want some tools people use to monitor activity of packets outgoing etc, ethereal, wire shark etc.
No problem. You can also consider Windows Sysinternals tools (Procmon is especially informative though it generates volumes of data). We also use sandboxes to look at these. There is a commercial product, CWSandbox, which supports malware analysis.
JFYI
http//
And a low cost/limited freeware tool
http//
jaclaz