±Forensic Focus Partners

Become an advertising partner

±Your Account


Username
Password

Forgotten password/username?

Site Members:

New Today: 0 Overall: 36125
New Yesterday: 1 Visitors: 132

±Follow Forensic Focus

Forensic Focus Facebook PageForensic Focus on TwitterForensic Focus LinkedIn GroupForensic Focus YouTube Channel

RSS feeds: News Forums Articles

±Latest Articles

±Latest Videos

±Latest Jobs

Principles of Testing

Computer forensics discussion. Please ensure that your post is not better suited to one of the forums below (if it is, please post it there instead!)
Reply to topicReply to topic Printer Friendly Page
Forum FAQSearchView unanswered posts
 
  

tootypeg
Senior Member
 

Principles of Testing

Post Posted: Mar 18, 19 22:00

Just trying to compile a list of important elements involved in the testing of newly discovered artefacts in order to reliably interpret their content.

So to start, repeatability of results (consistency) is important to make sure that certain actions result in consistent outputs.

....but what else is a key part in testing. Particularly where we are talking about 'new knowledge' - artefacts which are newly discovered. How do we approach this reliably.  
 
  

athulin
Senior Member
 

Re: Principles of Testing

Post Posted: Mar 19, 19 08:36

- tootypeg
So to start, repeatability of results (consistency) is important to make sure that certain actions result in consistent outputs.

....but what else is a key part in testing. Particularly where we are talking about 'new knowledge' - artefacts which are newly discovered. How do we approach this reliably.


This sounds very much like a question about scientific discovery and modelling, and I suspect the main sources are found in that area.

However, bugs and other malfunction will be a part of your observation results, and can not necessarily be easily identified as such. These can be very difficult to fixate. (Experience: Microsoft OLE, which in its first generations did not work entirely according to specifications. It worked for Microsoft objects ... but they did not call external API, but used their own internal access points ... you basically had to sit in the lap of the developers to learn that ...)

Side effects of your observation may affect your results. (Anyone who tried brute-forcing a password, but where the authentication just locked up after ten false guesses know what I mean. Consistent results are consistent results after all. Those first 10 tries are probably just experimental error.) Some forms of tests may trip security event detectors or canaries.

Some functionality ... especially in security-conscious software ... may be intended to decieve. (or just not work. Experience: login form to an Windows encryption program. Most Windows application windows can be 'eavesdropped' on ... this couldn't. It wasn't a Windows window, it was something that looked like a Windows window, but was implemented is a different way. You can bang your head against a security-enabled application for a very long time until you realise that it is protecting itself against people like you.)

That is, the hypothesis that 'my tests won't affect operation' is not necessarily one that can't be tested until you've discovered what 'operation' really is.

I vaguely remember an SF novel, where an alien artifact was scientifically examined at some point. The last hypothesis was that it probably was intended for amusement or recreation, and was basically an alien counterpart to a Las Vegas slot-machine. That is, it was basically random, and any 'results' were also random. (Was it 'Roadside Picnic'?)

And Journal of Irreproducible Results may be something to think about: while it (now) is just humour, I believe it started from the observation that irreproducible results are also results, but we just don't know what to do with them, as scientific publication does not accept a 'we failed'. But the next team to attack the problem may need to know what has been tried and didn't work.  
 

Page 1 of 1