Video recovery
Videos have become an important part of our lives but unfortunately can get deleted or lost. As described below, due to the way that videos are saved on memory chips, many software programs will not recover them correctly. GPR solves that problem for GoPro and a growing number of drones and video cameras, such as Canon EOS.
Why video is complex to recover?
A common way to recover files is to find a file start, and then assume that the file data is sequential. For many items such as photos this works, but typically not for videos. Most video cameras store the file in fragments on an SD card. On a FAT 32 memory chip, the address of each fragment of a file is stored in the File Allocation Table (FAT). When a file is deleted, so is the allocation data in the FAT. This means there is no way to 'undelete' the file because the location of each fragment has been lost.
Why are GoPro Hero files stored in many fragments?
The reason for this is that camera records both low and high resolution video at the same time. The data stream of clusters on the memory chip could look a bit like below
<high> <high> <high> <low> <high> <audio> <low> <jpeg> <high>
Each element is logically written into it's own file. Thus on a working chip, you will just see the files. However, remove the FAT (such as when deleted) and all you have is a series of clusters with data from one of the above streams. The length of each run of clusters is different for every video, and so is the sequence. Hence, finding the file start and hoping for sequential data does not work.
How does Gopro Recovery solve this problem?
The problem to solve is a bit like having a jigsaw puzzle and all the pieces upside down. Some pieces you can pick first, eg 4 corners, and the the edge pieces. The rest would be a bit of a guess - though one can assume that typically, but not aways, the pieces might still be in the correct area of the memory chip. Selecting these unmarked fragments in the correct order is where years of development has been spent to get the best possible results. The GPR program analyses each cluster of data and then reassembles each video using the correct clusters. This results in the original playable files. Thus the files are recovered, not repaired. Many reconstructed files actually have several hundred fragments and thousands of clusters.
The problem is much like a jigsaw puzzle - all the pieces, but jumbled up, as in the pictures below
GPR will in effect reconstruct the pieces on the left to the image on the right. A few missing pieces will not stop the process.
The program restores both .MP4 files and .LRV, (Low resolution video).
On some chips, there may have been damage, making reconstruction impossble - ie the original data no longer exists. For these devices, the forensic version has some extra tools. Clusters are scanned but the meta data is then created from scratch. It means that small fragments of video can be viewed. The downside is that sometimes many small files can be created, instead of a single large file. The positive side is that even small fragments can be useful. At no time is any video data changed, so it remains forensically secure.
Drone video recovery
There are many drone cameras that store data in a similar way to GoPro, ie both high and low resolution streams. GPR will automatically detect and handle many of these cameras, processing each stream separately so that the final restore video is both complete, and plays correctly. The process will often reconstruct 10s to 100s of separate video fragments.
Canon camera recovery
Canon cameras do not have the problem of multiplexed video streams, but typically the video is not recorded in sequence. There are two main variations. Some cameras record all the ftyp (header) and moov (frame pointers) headers at the start of the memory chip, and all of the video data (mdat) further down the chip. This means that the two very separate sections have to be associated with each other and recovered.
The second main variation is that the video (mdat) is recorded first, and then the ftyp/moov atoms are recorded last. Logically this will be taken care of with the FAT, but the FAT information is lost when a file is deleted, or chip reformatted.
With all variations there is the issue that the data may not be continuous. This can be caused by memory chip fragmentation after deleted previous files, or due to JPEG photos beimng saved at the same time as the video. GPR will resolve all these possible problems.
Nokia
A variation on a Nokia phone was seen recently. The video clusters were stored in reverse order. Normally a file (of any sort) has cluster in sequence, eg 10,11,12,18,20,21. The Nokia phone had videos recorded 30,29,28,27.
Main elements of a video file
There are three main sections on a video file, ftyp, moov, mdat. These are called atoms.
The ftyp atom is always at the start of the video. It is short, maybe 24 to 40 bytes in length and has a very brief technical description of the video
The mdat atom is where all the data is stored. This will be a sequence of video frames and audio frames.
The moov atom is largely an index for the video and audio frames, but also basic information on the video. This is a very complex structure, but GPR will recreate it in the forensic version, if the original has been lost/deleted.
A further complexity is the order these atoms are logically saved within an .MP4 file. It can be either of the following orders.
<ftyp><moov><mdat> or <ftyp><mdat><moov>
The physical order on the memory chip can be different again! Nothing is fixed, but GPR software will work with any combination.
Why video is complex to recover?
A common way to recover files is to find a file start, and then assume that the file data is sequential. For many items such as photos this works, but typically not for videos. Most video cameras store the file in fragments on an SD card. On a FAT 32 memory chip, the address of each fragment of a file is stored in the File Allocation Table (FAT). When a file is deleted, so is the allocation data in the FAT. This means there is no way to 'undelete' the file because the location of each fragment has been lost.
Why are GoPro Hero files stored in many fragments?
The reason for this is that camera records both low and high resolution video at the same time. The data stream of clusters on the memory chip could look a bit like below
<high> <high> <high> <low> <high> <audio> <low> <jpeg> <high>
Each element is logically written into it's own file. Thus on a working chip, you will just see the files. However, remove the FAT (such as when deleted) and all you have is a series of clusters with data from one of the above streams. The length of each run of clusters is different for every video, and so is the sequence. Hence, finding the file start and hoping for sequential data does not work.
How does Gopro Recovery solve this problem?
The problem to solve is a bit like having a jigsaw puzzle and all the pieces upside down. Some pieces you can pick first, eg 4 corners, and the the edge pieces. The rest would be a bit of a guess - though one can assume that typically, but not aways, the pieces might still be in the correct area of the memory chip. Selecting these unmarked fragments in the correct order is where years of development has been spent to get the best possible results. The GPR program analyses each cluster of data and then reassembles each video using the correct clusters. This results in the original playable files. Thus the files are recovered, not repaired. Many reconstructed files actually have several hundred fragments and thousands of clusters.
The problem is much like a jigsaw puzzle - all the pieces, but jumbled up, as in the pictures below
GPR will in effect reconstruct the pieces on the left to the image on the right. A few missing pieces will not stop the process.
The program restores both .MP4 files and .LRV, (Low resolution video).
On some chips, there may have been damage, making reconstruction impossble - ie the original data no longer exists. For these devices, the forensic version has some extra tools. Clusters are scanned but the meta data is then created from scratch. It means that small fragments of video can be viewed. The downside is that sometimes many small files can be created, instead of a single large file. The positive side is that even small fragments can be useful. At no time is any video data changed, so it remains forensically secure.
Drone video recovery
There are many drone cameras that store data in a similar way to GoPro, ie both high and low resolution streams. GPR will automatically detect and handle many of these cameras, processing each stream separately so that the final restore video is both complete, and plays correctly. The process will often reconstruct 10s to 100s of separate video fragments.
Canon camera recovery
Canon cameras do not have the problem of multiplexed video streams, but typically the video is not recorded in sequence. There are two main variations. Some cameras record all the ftyp (header) and moov (frame pointers) headers at the start of the memory chip, and all of the video data (mdat) further down the chip. This means that the two very separate sections have to be associated with each other and recovered.
The second main variation is that the video (mdat) is recorded first, and then the ftyp/moov atoms are recorded last. Logically this will be taken care of with the FAT, but the FAT information is lost when a file is deleted, or chip reformatted.
With all variations there is the issue that the data may not be continuous. This can be caused by memory chip fragmentation after deleted previous files, or due to JPEG photos beimng saved at the same time as the video. GPR will resolve all these possible problems.
Nokia
A variation on a Nokia phone was seen recently. The video clusters were stored in reverse order. Normally a file (of any sort) has cluster in sequence, eg 10,11,12,18,20,21. The Nokia phone had videos recorded 30,29,28,27.
Main elements of a video file
There are three main sections on a video file, ftyp, moov, mdat. These are called atoms.
The ftyp atom is always at the start of the video. It is short, maybe 24 to 40 bytes in length and has a very brief technical description of the video
The mdat atom is where all the data is stored. This will be a sequence of video frames and audio frames.
The moov atom is largely an index for the video and audio frames, but also basic information on the video. This is a very complex structure, but GPR will recreate it in the forensic version, if the original has been lost/deleted.
A further complexity is the order these atoms are logically saved within an .MP4 file. It can be either of the following orders.
<ftyp><moov><mdat> or <ftyp><mdat><moov>
The physical order on the memory chip can be different again! Nothing is fixed, but GPR software will work with any combination.