in our case we need to search one megabyte long hex value block in forensics image
how can we do this ?
Why does the block you are searching for need to be 1MB in size?
I assume the image you are searching this block for is much much larger, e.g. Gigabytes in size.
Could you just pick the first 20 characters and search for all hits in the image.
20 characters should be a fairly unique string. The small number of hits means you can then check potential matches by hand.
You could also then extract 1MB of data from the byte offset at the start of the hit, then use any number of tools to do a binary diff on the two 1MB files.
If you need more detailed steps let me know.
Write a simple program.
It may not be optimised but you know what is going on.
in our case we need to search one megabyte long hex value block in forensics image
how can we do this ?
You probably can't – at least not easily. Ordinary search tools (you haven't said what tools you *are* using), tend to have been designed for more limited jobs, and often have a limit at 512 bytes (hard disk sector size) or 4096 (common cluster size) or something like that. Those limits are not always well documented, so you need to validate your tool before using it with such outsized search patterns.
Writing your own program is always a possibility, but similar restrictions apply. Standard search functions usually have similar limitations themselves.
Then, there's the question if you have described your problem accurately 'search in a forensic image'. Are there any additional restrictions? Can the pattern start in any byte position, or only at sector offsets, for example?
If you are looking for a 1 Mbyte *file*, you have to take file fragmentation into account, which is another kettle of fish. Finding existing files are probably best 'searched' by hashing hash the prototype, then hash all files of the same length – if the hash matches, you can be reasonably certain you have a match. Finding deleted files … well, that's another problem again. But I assume this is not what you're looking for.
If the problem is locating consecutive sectors (i.e. matches must be on sector boundaries), it may be possible to transform the problem into searching for sector hashes split the search pattern into sector-sized chunks, hash each, and concatenate the resulting sequence of hashes into a new search key, and then, do the same thing for the image, then make the search on that.
Now you have reduced the problem from looking for a 1Mbyte pattern search into looking for 2048*hash length search . As you want this to be fast, use a very fast and very short hash/checksum algorithm – MD5 is probably overkill – say, Adler-32 or CRC-32, which means 2048*4 - 8192 bytes search pattren. You may even be able to reduce it further, with some increase in the likelihood for a false postive. But as you will verify each hit in the original image, any false positives will not affect the result materially, unless they become too many.
This would mean writing one fairly simpe tool for reading a sector file, and spitting out the corresponding hashes. Then you need a search tool that works on 8192-byte patterns – though you may need to write that as well.
For a small image and a job that isn't likely to be repeated, all this is probably overkill, though.
You probably can't – at least not easily. Ordinary search tools (you haven't said what tools you *are* using), tend to have been designed for more limited jobs, and often have a limit at 512 bytes (hard disk sector size) or 4096 (common cluster size) or something like that. Those limits are not always well documented, so you need to validate your tool before using it with such outsized search patterns.
I cannot see this being a problem, even if fragmentation is considered, lest we forget the good Lord invented hashing… specially if the 1MB is cluster aligned.
I second writing a code. Developing such code should take you no more than a few hours even for a beginner. It will be slow but will work.