• 43 Posts
  • 228 Comments
Joined 1 year ago
cake
Cake day: June 9th, 2023

help-circle





  • Slowly trying to learn sh while using mostly bash. Convenience is nice and all, but when I encounter something like OpenWRT or Android, I don’t like the feeling of speaking a foreign language. Maybe if I can get super familiar with sh, then I might explore prettier or more convenient options, but I really want to know how to deal with the most universal shell.



  • So Flash memory works in blocks called pages. The pages contain a header that ends in a few bytes that says what the rest of the page maps to.

    If the file was encrypted, you’re probably SOL. If it was not encrypted it may be possible to to recover some parts of the files. This is extremely advanced level data recovery. I only know the abstract basic principals and would likely struggle to figure this out and recover my own stuff if I ever needed to do this. I’ve only programmed microcontrollers and flash memory devices.

    A micro SD card contains a small microcontroller and some blocks of flash memory, although the microcontroller is transparent to the user and operating system… unless hacking with needle probes in a lab.

    So here’s the basics. Writing flash involves taking an entire Page of memory and zeroing it first. There is a tiny voltage booster circuit on the card that allows the page to get pulsed up and down in voltage a few times in order to completely zero the entire page without any remaining residuals. Once this is done and the entire page has been zeroed, only then is it possible to write the data into the bytes of the page.

    If you want to change a single byte level value in an address that already contains a value, first the entire page is copied to a blank page in another location, then the old page is pulsed a few times, then each value is transferred back into the old page except that the new value that needed to be changed is now set to the new values.

    This is the proper way to write flash at a basic level. If the power is lost in the middle of this cycle, the worst case scenario is that the new updated value was not written. The page in question should never be “missing” because the header record should always point to either the original or copied page. One of the two should always be present and complete… in a proper setup. Obviously, it might be faster to simply use some RAM to hold the page, erase the old page and rewrite it. I have no idea what size pages are in modern SD cards, but on hobby class microcontrollers I have used the pages were 4096 bytes, IIRC. My understanding is that most SD cards use an 8051 clone micro, so it is probably a similar size.

    So here’s the thing, the bulk of the data is always there. Somewhere deep down inside you likely already knew this. It is why you’re supposed to overwrite an entire drive instead of the “quick” erase in most formatting tools. The quick erase is simply deleting a tiny header file that says what exists where on the drive. Similarly, some part of your SD card there is a page or few where the header has been screwed up. Your OS is looking at this header info and seeing a mismatch of garbled junk and saying f-that bs.

    Generally, recovery would involve dumping the raw contents of the flash memory as hexadecimal, being super familiar with what you’re looking at and knowing how to find the page that is causing the error. Generally I assume you’d need to replace the bad page with a good header and it would then work. There are services for this kind of operation; data recovery. In practice, this has a few more layers of complication. Pages can be placed in different locations that enable wear leveling so one area of memory is not over utilized. There is also a table of bad blocks/pages that the micro knows to skip, and there is usually a bit or address in the page that is used to detect errors that may have occurred.

    This is pretty much everything I know on the subject. Hopefully it helps you understand the abstract nature of what is happening. In the simplest of terms, flash memory is like writing a long essay with an ink pen and where you can not make mistakes or use whiteout. If you need to make a change, you must write out the entire page all over again. This process is what is so time critical that you must “eject” the drive.





  • It follows the first to market principal in many of the most active communities, and it is the most federated instance. Many instances that are not federated with each other are federated with .ml. You still won’t see those comments between instances. Like from my main account here on .world, I can’t see hexibear or beehaw stuff, but from my .ml account I can see them. I have accounts on many instances in order to help federate new communities and to check biases/instance behaviors.

    I came over a few days before the rexodus and subscribed to the active communities before the influx. That sub list is still centered around the most active communities, and the majority of those are from .ml and before I joined Lemmy.

    Instances all have different flavors. I don’t like using my .ml account as a main. I’ve tried it. But I find they are the center of the most interesting and productive conversations for a more broad audience, while Beehaw has the most positive and friendly conversations overall. The main benefit to .world is the speed of connectivity, general audience scope, but with a strong anti asshat policy.



  • Not sure. There was some controversy with some of the devs making alt front ends and admins complaining about the slowness. I’ve seen mention of one of the two devs learning Rust just to participate. So it is not entirely a walled garden. The front end devs wanted to make an alt from scratch but in something like JavaScript although I don’t recall the details exactly. There were a lot of red flags related to privacy and understanding the community at large in the posts I saw from them. When asked why they weren’t adding pull requests with Rust in order to address their complaints I got no reply.

    All that said, I’m no dev. I can read in to around half the code I come across if I really try, and can successfully modify maybe half of that if I spend a few days on it, but I suck at clever code and the DRY cult types. I haven’t tried to look into Lemmy in any depth beyond figuring out the basics.


  • The Lemmy algorithm:

    https://join-lemmy.org/docs/contributors/07-ranking-algo.html

    The instances communicate between each other using special bots in the background that transport info between instances. Dot world is too big and makes too many requests to other instances. Most instances are reducing the number of transport bots back to dot world right now. There is supposed to be a fix in the next Lemmy version, but the code base is somewhat slow moving due to only two devs and it is written in Rust. Rust is a hard language like C, and not too many here are able to contribute to it, even though it is like the new gold standard of code.

    So you might see a delay between posting and replies or the interaction may come in bursts that correspond with the transport bots carrying content between instances as the host admin have configured their instance.




  • I found a Python project that does enough for my needs. Jq looks super powerful though. Thanks. I managed to get yq working for PNG’s, but I had trouble with both jq and yq with safetensor files. I couldn’t figure out how to parse a string embedded in an inconsistent starting binary, and with massive files. I could get in and grab the first line with head. I tried some stuff with expansions, but that didn’t work and sent me looking for others that have solved the issue better than myself.