Contradictory test ...
 
Notifications
Clear all

Contradictory test results:UFED vs XRY vs SIMcon  

Page 1 / 2
  RSS
yunus
(@yunus)
Active Member

I had a small test covering common SIM examination tools, UFED, XRY, SIMcon.

The results were more contradictory than expected, sometimes even opposing. The contents, time stamps and status of the SMS messages were different from each other.

See the results at http//www.dijitaldeliller.com/yazilar.html

Click on the link "english version the test report" on the top of the page. The report is in pdf file.

Regards,

Quote
Posted : 07/05/2013 11:24 pm
mscotgrove
(@mscotgrove)
Senior Member

I am not familiar with any of the tools you are looking at.

Tools are only 'quick ways' to get results and can produce false positives - or just incorrect results. The ultimate test is a hex editor and a lot of knowledge. That way you may get faith in a tool, or know it limitations.

ReplyQuote
Posted : 08/05/2013 1:29 pm
(@keydet89)
Community Legend

I agree with Michael.

One of the biggest fallacies that I hear on a regular basis is analysts saying that having multiple tools to do the same thing allows you to validate your findings. This is patently incorrect.

If you understand the data structures that you're looking for/at, then you can choose the appropriate tool for the job, and then validate your findings with a hex editor.

The fact is the multiple tools can be "wrong". Some tools are not written by analysts who do the work and understand the need, but are instead written by developers who have little understanding of what's needed and do not do the actual work.

ReplyQuote
Posted : 08/05/2013 6:05 pm
 jhup
(@jhup)
Community Legend

So, to expand on that - what does the raw data look like (do not post, just look)?

(A side note - your data is not anonymized sufficiently. ex. line 3 in first table, phone number ends in 55.)

ReplyQuote
Posted : 08/05/2013 9:08 pm
(@paulsanderson)
Senior Member

You need to anonymise your data - it is trivial to get at the numbers under the white box you have layered.

The first number is +9054?????705 (added ? to protect the guilty/innocent)

ReplyQuote
Posted : 08/05/2013 9:24 pm
(@paulsanderson)
Senior Member

So, to expand on that - what does the raw data look like (do not post, just look)?

(A side note - your data is not anonymized sufficiently. ex. line 3 in first table, phone number ends in 55.)

Oops cross post - but I would change "anonymized sufficiently" to "anonymized at all".

ReplyQuote
Posted : 08/05/2013 9:25 pm
(@jaclaz)
Community Legend

If I may, and from what I can understand, you all are saying the same thing (including the OP).

No software should be trusted "blindly", but, as I see it, Yunus did a nice job ) in highlighting a possible "real life" issue.

What is the practical suggestion?

  1. Do not use any third part software and write your own tool.
  2. Use multiple softwares and delve deeper if there are differences between results.
  3. Test as deeply as you can all available software, then choose one and only that one, because you already know which "quirks" it may produce.
  4. Other (please specify).
  5. [/listo]

    If #1, then all software houses could close down, and each forensic examiner will have to write, validate and eventually "defend" in court his/her tool against the findings of the expert witness of the other party. who also wrote his/her own tool and has exactly the same issue.

    If #2, then all forensic investigators should test the data/SIM/whatever against *all* available tools, freeware or Commercial and "hope" that no inconsistencies are found. (as one of the tools may be "right" in one specific point but "wrong" on another)

    If #3, the risk of a peculiar artifact not having being tested or tested properly seems to me rather BIG.

    I would say that all three approaches above are either very problematic or very non-productive.

    What is option #4? ?

    jaclaz

ReplyQuote
Posted : 09/05/2013 12:29 am
trewmte
(@trewmte)
Community Legend

Yunus, do you have the message header/s for the original text message/s from the target SIM Card that you can post so that we can start at the beginning.

The GSM standard reference template

Example of output message header data looks like this

More Messages To Send No
Status Report Indication No
Reply Path No
Originating Address Length 0C
Originating Address type 91
Type of number International
Numbering plan identifier E.164
Originating Address 44798021XXXX
Protocol Identifier Default
Data Coding Scheme GSM Default Alphabet
SC Timestamp 10214201358500
decoded 01/12/24 105358
Time Zone GMT+0.00H
User Data Length 4C
decimal 76

This is an obvious question but worth asking anyway. Can you also confirm whether you used one single test examination computer or whether each SIM card reader was attached to different computers at the time data was 'extracted' and 'harvested' from the target SIM card?

Can you confirm whether the original text message/s were saved directly to the target SIM card or saved in the handset and then later saved to the target SIM card (eg in terms of Message Class 0, 1, 2 and 3)?

ReplyQuote
Posted : 09/05/2013 1:00 am
(@keydet89)
Community Legend

If I may, and from what I can understand, you all are saying the same thing (including the OP).

I have some thoughts that I'd like to share, but I'm going to move this to another thread…

ReplyQuote
Posted : 09/05/2013 10:55 pm
(@jaclaz)
Community Legend

I have some thoughts that I'd like to share, but I'm going to move this to another thread…

Which is this one
http//www.forensicfocus.com/Forums/viewtopic/t=10579/
(just to keep things as together as possible)

jaclaz

ReplyQuote
Posted : 10/05/2013 3:11 am
athulin
(@athulin)
Community Legend

I would say that all three approaches above are either very problematic or very non-productive.

What is option #4? ?

Perhaps just what the OP actually did test, observe a problem and publish (informally or formally). That's what 'security researchers' do in closely related areas – test software, identify security problems, and publish. (Some of their discoveries even get patched … ) The difference seems to be that much of DF software is closed source and expensive, so not everyone can do the actual testing, and not everyone who has access can create the tests themselves.

Brian Carrier's initiative DFTT identified 'Testing in the public view' as important – it seems to be just what is being discussed here. While the tool being tested may be expensive, and so out of reach to some, the test design itself and any test data should be easier to get involved with.

So … are there any public test suites/test images for this particular area? If not, is it possible to build some? What would it require? What does it need to adapt to – different releases of Android or some other software? What questions should it answer? Etc.

Even if only a half-decent design/suite is produced, it's a platform that can be improved and extended. If everyone has to reinvent the wheel, we'll never get out of this particular ditch, unless the tool makers magically get their collective act together.

ReplyQuote
Posted : 10/05/2013 7:40 pm
yunus
(@yunus)
Active Member

I'd like to thank everyone for comments.

To trewmte; I do not have the header/s for the original text message/s from the target SIM Card from UFED or XRY. But I have the following details from SIMcon extraction

TP-MTI SMS-DELIVER
TP-MMS MORE MESSAGES WAITING
TP-RP NO RP
TP-UDHI THERE IS NO INFO THAT GOES BY THIS NAME IN THE EXTRACTION
TP-SRI STATUS REPORT SHALL BE RETURNED
TP-OA 90…………(THE SENDER'S PHONE NUMBER)
TP-PID MOBILE-MOBILE
TP-DCS-CODING UCS2
TP-DCS-CLASSIMMEDIATE DISPLAY
TP-DCS-CLASS(EMPTY)
TP-SCTS 15 aug 12 190928 gmt+03.00
TP-UDL THERE IS NO INFO THAT GOES BY THIS NAME IN THE EXTRACTION

There are also other details in addition to the above ones. Maybe they include the above missing details under different name. The report is updated with more anonymization and SIMcon extraction message details is added. http//www.dijitaldeliller.com/yazilar.html - Click for the english version of the test report.

I have used different readers on different computers. XRY and UFED has it their own readers. And with SIMcon, I have used and SIM card reader. And the card reader was attached to different computers at the time data was 'extracted' and 'harvested' from the target SIM card.

I do not know whether the original text message/s were saved directly to the target SIM card or saved in the handset and then later saved to the target SIM.

Regards,

ReplyQuote
Posted : 10/05/2013 9:59 pm
trewmte
(@trewmte)
Community Legend

Hi Yunus, thanks for checking and your replies. I will study the data you have given and offer some suggestions.

I was raising different observations in my post to see whether there would be other reasons other than the way each program extrapolates the data pointing to the conflicts you mention in your original post - e.g. are the programs you are using; can sometimes not reproduce the same output.

In part, you may have to accept that due to the way each programmer has written their SIM reading software that it may produce results the programmer personally thought was relevant as opposed to faithfully following the standard. By way of illustration you recorded

TP-UDL THERE IS NO INFO THAT GOES BY THIS NAME IN THE EXTRACTION

UDL is important when checking to see the length of the message sent actually matches the length of data in the production of a received message. As you know, this is where you 'might' see conflict between the actual message sent and potentially an altered message residing in the inbox.

I would like to look into your matter further and ask questions. However, this equally requires your time, if you are willing and able to give it?

Out of curiosity has XRY, UFED or SIMcon contacted you, yet, to offer an explanation?

ReplyQuote
Posted : 11/05/2013 2:24 am
RonS
 RonS
(@rons)
Active Member

Yes, I did (PM) and I am waiting for yunus reply.
There might be several reasons and we are checking this.

ReplyQuote
Posted : 11/05/2013 12:09 pm
yunus
(@yunus)
Active Member

Hello trewmte,

Yes, you can ask further questions and I am willing and able to give it, however, if futher details requires the actual SIM card or generating a log file by re-examining it, I may not be able to provide details as the actual SIM was already sent to the requestor.

By the way, UFED and PARABEN (who purchased SIMcon)has contacted me.

UFED asked for generating a log file, however, the SIM was already sent to the requestor, so I can not generate it.

PARABEN said they purchased SIMcon, integrated the code into device seizure and asked if I would be willing the test the Device Seizure.

Regards,

ReplyQuote
Posted : 11/05/2013 1:00 pm
Page 1 / 2
Share: