Hi Veronica! I am writing a book chapter on memory storage in cyber-physical systems.

Would you like to join me on this venture? The book is going to be published by IET. The 1st reviewed version will be available tomorrow. If you accept, then I can send you the text so that you can criticize and add new material (including your papers)!

Please, email me ASAP: [email protected]

Vania

Is that possible to provide a dynamic model in these systems, e.g. first-order or second-order? If yes that would be great to study them in different working conditions.

any simulating tool for blockchain-based storage and new areas for research work?

Hi Veronica! I am writing a book chapter on memory storage in cyber-physical systems.

Would you like to join me on this venture? The book is going to be published by IET. The 1st reviewed version will be available tomorrow. If you accept, then I can send you the text so that you can criticize and add new material (including your papers)!

Please, email me ASAP: [email protected]

Vania

Erasure-coded storage on disks in Ceph might be a good topic to look at. You can probably assume S3-compatible RGW storage with a default 4MB block size, not RBD block (my impression is that RBD over EC over disk is a really bad idea) or CephFS. We've done some limited experiments indicating a speedup for write and possibly a major slowdown for read due to seek overhead, since the EC implementation breaks up the 4MB blocks into much smaller chunks.

Asif Ali Laghari For blockchain-based storage, I'm interested in knowing which codes and settings are implemented or (in future plans), A list of potential schemes (an the reason why a setting is a candidate RS(x,y) because x, y blah blah...and we need protection for attack z...). My model will help to decide which is more appropriate to the specific use cases. Blockchain or similar is a very different from cloud storage, thus setting changes too, and repair policies? I need concrete examples.

I think cold storage (tape, optical, dna, powered-off hdds etc) would be as important as blockchain-based and ssd and memory-based hot storage. Coding, data allocation, reliability modeling, all pose different set of risks, tread-offs and requirements...

My experiences are more from the world of hardware problems. I'm dealing with data recovery. I recover data from individual media and from complex systems, such as RAID. The nature of my work shows that I have contact with emergency situations. I often deal not only with a primary fault, but also with secondary faults caused by incorrect system administrator actions.

The most common mistakes include attempts to restore the efficiency of the environment that has been damaged. Very often during these attempts data on the media is overwritten. For example, you can restore data or run automated system repair procedures to overwrite data. The problem occurs when, for example, the backup contains errors, and we do not have the possibility to return to the state before it was restored.

it is a serious mistake to replace too many discs in the array that have failed (for example, for RAID5, a maximum of 1 disk can be disrupted, for RAID6 - a maximum of 2 ...). In the event of a failure of too many disks, it is necessary to disable the matrix from operation and attempt to recover data from damaged disks and assemble a matrix from what has not been broken + from what has been recovered.

Rebuilding a matrix with a large number of new empty disks leads to data loss.

Another category of errors is negligence and lack of reaction to errors and warnings. This is a situation where a quick and proper administrator response could prevent a failure. Sometimes it is not the fault of the administrator, but the procedures, the scope of decision making, and the budget. Sometimes it is a relatively small amount of money, but the administrator has problems with obtaining approval for their release. Until the failure of the entire system, the purchase of the necessary component does not seem to be a priority.

There is really no system that can not crash. And it is very difficult to predict all scenarios, the more so because they often have to include random events and human errors that you can not influence. I often talk to people who know very well what error they have made, so the cause of this error is not the lack of competence, but the moment of human weakness. Redundancy, procedures and protections can only reduce the risk to a level where the vast majority of systems work long enough with sufficient reliability.

It is possible that on the scale of all correctly operating systems and all correct responses to failures, situations that I encounter are very rare exceptions, but they are.

Paweł Kaczmarzyk do you have any procedure to recover information from disk arrays in which the maximum number of disks has failed (for example if 3 disks of a RAID6 failed)? In those cases, is it possible to recover data with forensic tools? Besides, I wonder if there is any statistic available, something like "with this tool you can recover X % of the data stored in the array".

In the event of a disk failure, the data is usually still on these disks, but it is not available to the user. To retrieve information from the RAID array, it is necessary to read the contents of the damaged disks. In our example (RAID6 - damaged 3 disks) it is often enough to read the contents of one of these disks - the one that is the easiest to read. If you can get an exact copy of this disk, often just to recover the data, it is enough to substitute it to the matrix. In the case of electrical damage, you can often repair the disk.

It is often the case that we can not get an exact copy of the minimum number of disks to run the matrix. Then we try to make the best copies of all disks and assemble the matrix from the fragments obtained, filling out the gaps on the basis of redundancy. In principle, it can be done using a hex-editor, but it is very labor-intensive. That is why we also use other tools - eg DMDE, R-Studio, PC-3000. These tools automatically analyze the logical structure of RAID arrays and file systems. Sometimes they can reproduce the entire logical structure themselves, and sometimes you have to improve something manually. But it is much less work than if the entire logical structure should be manually reconstructed.

As for the effectiveness of data recovery - it depends to a large extent on how much of this data is actually recoverable and how much was lost. The reasons for data loss at the sector or byte level can be considered in two categories:

1 / - physical causes - mechanical damages to the disk surface (eg as a result of contact with the head) or magnetization errors (one of the reasons may be unstable electric current feeding the disk), damage to pages in flash-NAND memory devices, etc.

2 / - overwrite - which may be a consequence of attempts to automatically repair the logical structure of the file system (eg chkdsk) or human errors when trying to start the system ..

From the point of view of data recovery efficiency, physical damages have the advantage of protecting data against user errors causing overwriting.

Sometimes we deal with situations where we are not able to read the data, although they are potentially available in some way. This applies in particular to mechanically damaged surfaces of hard disk disks. There are situations when we can not read the undamaged part of the plate, because before we do it the heads are destroyed by passing over the damaged fragments of the surface.

Another factor that affects the effectiveness of data recovery is the knowledge, skills and determination of the man who recovers this data. Most recovery companies are limited to following the procedures described in the instructions. This is the most economically justified behavior. But you can often get better results if you put in a bit more effort to analyze and understand the nature of the problem.

Various types of tools facilitate and speed up the work, but they do not do it themselves. I have been able to recover data efficiently using relatively simple tools in situations where international corporations with all available tools have found that recovery is impossible. Therefore, it is difficult to talk about performance statistics for individual tools. The adequacy of the tools used for a specific defect is much more important.

Another statistical problem is the translation of the results of the physical reading of the disk contents to the recovered user files. Sometimes we have a physical reading of 70-80% and this is sufficient to fully reproduce the logical structure and all user files. But there are also situations where we have 96-98% of physical reading, but surface defects are so arranged that they damage our logical structure or half of the most important documents. For the user, it is not important that Windows system files were recovered. His documents are important. This problem is particularly significant in the case of SSD. The immediate cause of SSD damage is very often write errors. And usually these are errors regarding recently saved data. This is often the most important user data that has not yet been backed up.

Do not believe in the statistics provided on the websites of companies recovering data. Most of these companies do not have any statistics at all. They often enter their effectiveness at the level of 76% - because that's what wikipedia gives - or above 90% - to encourage customers to use the services of this particular company. Even in the case of companies maintaining statistics of their effectiveness, the methodology of these statistics is often subordinated to marketing goals.