I know these comments are going to be full of people touting the virtues of having backup drives, NAS, or other high level data protection, but am I the crazy one? Knock on wood, I know nothing lasts forever, but I have decade+ old usb drives still going strong. How do they burn through so many externals?
I think selection bias is part of it, we tend to hear from the folks who run into issues more than the folks who don’t. I also think a drive that sits on a desktop or in a drawer most of the time in an air-conditioned house will last much longer than one that’s often thrown into a bag and transported in vehicles, airports, etc.
I love fake product reviews. You can see the marketing speak just dripping off of them. I swear people in marketing can’t control themselves when it comes to speaking like an ad.
I’ve been telling people for years that the entire 21st century is at risk of being a lost century. Even personally I can’t guarantee my data will be with me 20 years from now even though I back it up. If you care about a photo or document, print it and throw it it a box. As I get older I find more of an obsession with physical media from a preservation point of view. Because I know my books and pictures will be around 50 years from now. Digital files not so much.
Even data from the Apollo missions was found to be either degrading (tapes) or the formats were forgotten and the systems that could read them were gone. They had to do research into rediscovering how to read the data and hunt around for the few antique systems remaining to read the tapes.
I’ve been telling people for years that the entire 21st century is at risk of being a lost century. Even personally I can’t guarantee my data will be with me 20 years from now even though I back it up. If you care about a photo or document, print it and throw it it a box. As I get older I find more of an obsession with physical media from a preservation point of view. Because I know my books and pictures will be around 50 years from now. Digital files not so much.
LOCKSS and KISS, though. Flash chips don’t last forever but are pretty durable, and so are optical media as long as they’re the right material. SSDs decay and HDDs fail, but for magnetic platter media even if the head or motor crashes there’s always the old magnetic microscope in a pinch. USB’s not going anywhere, and if you have four or five copies that you don’t completely neglect and don’t store in the same physical place, presumably you’ll have the chance to notice and take corrective measures if any of them start failing or are at risk.
I don’t actually know that an individual book or picture will still be around in 50 years; Fire, flooding, insects, acidic paper, low-quality ink maybe— Digital stuff’s fragile, but so is physical stuff. Stick it in the attic, and the heat’ll speed up any chemical reactions and probably make it cozier for insects; Stick it in the basement, and the condensation will get you mildew and rot. By contrast, having a flash drive accidentally survive a trip through a washer and dryer is a pretty common occurrence, and I’ve yet to lose a drive even with that level of negligence. Material compatibility’s one of the very most basic parts of a set of very precise manufacturing techniques, tin whiskers seem pretty rare these days, the really scarily insidious stuff like hydrogen embrittlement is super improbable, and most biological forms of decay haven’t adapted to eating cured epoxy and monocrystalline silicon yet.
At least I sorta know how a flash cell or hard drive platter is meant to be structured; Who knows what weird organic reactions and unstable or slowly diffusing molecules are happening in the pile of chemical pigments on a sheet of likely-acidic bleached cellulose and cheap ink or toner, and whether it will still be legible to human eyes in however many years? Plus, a printed photo or document starts fading the very instant it’s created, and it gets a little worse every time you touch it with sweaty human hands or look at it while exhaling moist human breath and corrosive enzymatic saliva droplets under a white LED lamp or G-type star shooting out ionizing UV rays. Digital failures tend to be catastrophic, but at least up until the moment it fails, you can make sure that it is the exact same picture or text— And you can make many, many copies very cheaply, all of them very physically durable compared to paper, and know that they are all the exact same picture and text.
That said, I absolutely agree with your overall assessment that most of the information in the early 21st century, including most of the public Internet/WWW, most likely either will be or already is… Maybe not technically lost, per se, given how much caching and saving happens on private clients, but certainly rendered inaccessible.
Ideally I’d really love to see a return of microfiche, actually, using modern polymers and metallization. I’ve been meaning to look into that for a while now. At a reasonable scale for optical viewing, you could fit… much, much more content than you might expect, and do it several times over, in an entirely reasonable number of pages. Your comment actually spurred me to finally think of a practical way of printing that— for years before, I’d been trying to idly figure out a process based on photomasks and nanoparticles suspended in resin, which had always felt like a very messy and tricky idea, but I just thought of another idea– So thanks for providing some inspiration there.
There is a will but there aren’t enough people with enough brain power to actually do the steps needed.
Should it endure? I don’t know, maybe the last few decades should be forgotten.
Crazy how a single event sometimes reminds people of bigger problems, huh?
Digital media, where we store basically everything we care about, is hugely, hugely volatile, unreliable, and fragile. But you never notice it until you’re reminded of it, and then you really notice it. This story reminded people of it.
The reminder to stay grounded is probably also healthy, but I do think you’re missing the point of this comment thread.
PCM, ASCII, and straight RGBA bitmap encodings aren’t going anywhere. By extension, derived formats like WAV, UTF-8, and word processor files and webpage HTML are mostly fine too. The formats are structurally simple enough that even if the associated file extensions were somehow to be forgotten, all you’d need to do to invent them again is hand the file to a bored nerd over the weekend.
I think you kinda got the BBC and NASA problems backwards. The BBC’s had a couple of prominent incidents where digital “preservation” that was supposed to be eternal couldn’t even be opened anymore after a couple of years, like their Domesday Book/Project application thingy. They’ve also lost a bunch of old shows, like early Dr. Who episodes, I think. NASA didn’t just forget how to read the Apollo tapes; they overwrote them to reuse the tapes, as was their standard practice at the time. The original signal and tapes were very HD (or analog), but most of the videos we have today are from the TV camera that they pointed at their own TV screen last-minute when they realized they didn’t have an adapter for broadcast— The equivalent of a grainy cell phone photo of a screenshot, basically.
The BBC and NASA incidents happened in an era before computers were a ubiquitous commodity product. So, everyone and their cat was basically inventing their own obscure single-implemention proprietary file formats at that time. Nowadays we have established technical standards, as well as formats that have already sorta stood the test of time based on their utility and simplicity— and millions of people who already know how to read them— so that particular vector for bitrot isn’t really as much of an issue anymore.
…That said, I think I sorta missed your point. What you’re really saying is that stewardship of digital records is much trickier and riskier than stewardship of physical records— and that results in stuff being lost. And that is absolutely true.
Many of the most culturally significant and creatively valuable works of literature, film, painting, and other forms of art and even science and engineering throughout history were widely panned, ridiculed, or simply unknown by their contemporaries.
“Will to maintain” comes from human people. And frankly, we’re vacillating, self-destructive, pleasure-seeking idiots with furthermore awfully finite practical limitations about what we can and want to have the will to do.
Lack of human eyes on something at a particular moment in time does not at all imply lack of its innate value. It doesn’t even guarantee lack of human eyes on it at a later date. Imagine if Da Vinci’s private notebooks, or Lovelace’s programs, or Van Gogh’s paintings, or Fermat’s humblebrag, or Melville’s writing and Dickinson’s poetry, or even Anne Frank’s diary, were saved on MicroSD instead of as physical hardcopy.
If there is a will to maintain it, it will endure. If there is no will, should it endure?
“Everything dies unless significantly effort and resources are constantly expended to maintain it” … Is not a world that I think I would prefer to live in.
…It would be an empty world and yet a violent world, filled with only the most trite and self-serving patterns designed and evolved to extract a profit in the briefest moment before they vanish. Even the stuff that does get “maintained” would have no fixed form, no verifiable truth, and no shared cultural experience or memory and heritage— Think of the streaming services that edit a film after it’s already been released, and imagine doing that to something like, E.G., Citizen Kane. Many of the things that we take for granted today— and probably most of our sense of cultural permanence, heritage, continuity, and memory— Might not really be possible in such a world.
And frankly, that’s going to bleed over into our politics too. Focusing exclusively on the single instant of the present because you no longer really have access to the past is a great way to make lots of selfish, impulsive decisions with disastrous consequence. It might even cause some degree of technological and cultural stagnation, as due to both psychological preferences and economic inequities, “will to maintain” is probably going to be heavily biased towards things that are already part of the status quo.
Physical media is at least somewhat inherently stable without constant maintenance on culturally relevant timescales, but digital media isn’t. At the same time that digital information technologies allow vastly greater quantities of valuable information to be assembled and disseminated, they also provide no practical way of passively preserving basically any of it. The invention of writing is fundamental to civilization, but the way it works is changing. Compared to the previous balance of capacities, we are now culturally capable of experiencing much, much more than we were before, but guaranteed to remember basically none of it— This is dangerous, or at least concerning and troubling.
Life is the perpetual fight against entropy, in all its forms. Meaningful information requires a lot of effort, work, insight, and feeling to put into words and pictures. Letting it be forgotten means forgetting who and what and why you are— Just because you can’t spare the CPUs and the technicians to keep its server up.
This comment thread wasn’t about that one videographer. It was about backup techniques and data preservation practices in general, and worrying trends in that direction.
OP: I’ve been telling people for years that the entire 21st century is at risk of being a lost century.
Got my gf a 2TB version. She also lost most of her files after 1 month of usage. She uses MacOS. But it’s probably to some degree of personal failure. So not sure if this is relevant.
So they just had this one drive fail and they decided to make a big news article about it? Hardware fails sometimes. Just RMA the thing and shut the fuck up about it. Go build a gaming PC.
The use case for these drives is always a single point of failure. It’s camera footage out in the field. You have to get it home before you have proper storage.
Do you have a dashcam for your vehicle? Do you have backups before you get the footage home?
“I put 3TB of irreplaceable data on a single drive, and want to blame anyone but myself for my data loss”
Go away with this garbage.
I personally have a NAS with 12TB striped over 3 drives, I sure wouldn’t blame WD if one drive failed and I lost everything.
E: this whole comment section is why tech illiterate people shouldn’t really comment on hardware failures like this. The only fact that is know is that the verge faced 2 drive failures and lost 3TB of data due to a lack of safe data storage practices. If they were tech literate they wouldn’t have lost any data.
The verge did not confirm the mode of failure, and therefore the second failure could’ve been completely unrelated to the firmware issue. Nobody knows anything, other than the verge needs to educate themselves on how to properly store irreplaceable data.
Correct. My next build will be redundant but given that my truenas pool is only storing movies, shows, music and porn, I don’t much care if I lose the contents due to a drive failure
The claim here seems to be that the product has an unusual failure rate, the manufacturer has acknowledged the original problem and released a fix, and it does not appear to be fixed. I don’t read it as a sob story about some reporter’s lost data.
Given the verges track record on tech reporting, i wouldn’t put faith in their journalistic integrity of a hit piece unless they show a bit more than “look, i lost a drive after they said they fixed the issue. They’re lying!”
I purchased a 2TB one of these SanDisk “extreme portable” drives in 2018, and 2 more 2TB drives in 2019. Purchased each one roughly 6 months apart. Knock on wood…so far no problems at all with any of the 3. But, drives do often fail (I’ve had several fail over the years). One general rule of thumb I have when shopping for drives is I never buy the model with the highest storage capacity for the product line. It’s just a dumb superstition I have, but it seems like the higher capacity ones (like 3TB and above) are the ones that have failed on me in the past.
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !technology@lemmy.world
This is a most excellent place for technology news and articles.
I use mine for desaster recovery.
Using tineshift to take hourly snapshots of my laptop computer.
I don’t think my laptop and the drive fail at the same time so I think my use case is safe even with these risky drives.
redundancy is key
redundancy is key
LOL
I know these comments are going to be full of people touting the virtues of having backup drives, NAS, or other high level data protection, but am I the crazy one? Knock on wood, I know nothing lasts forever, but I have decade+ old usb drives still going strong. How do they burn through so many externals?
I think selection bias is part of it, we tend to hear from the folks who run into issues more than the folks who don’t. I also think a drive that sits on a desktop or in a drawer most of the time in an air-conditioned house will last much longer than one that’s often thrown into a bag and transported in vehicles, airports, etc.
Right, we need more positive articles like “We just didn’t lose 3TB of data on a Sandisk SSD!.. Yep, the data is still there!”
I love fake product reviews. You can see the marketing speak just dripping off of them. I swear people in marketing can’t control themselves when it comes to speaking like an ad.
NAS w/ RAID…
I’ve been telling people for years that the entire 21st century is at risk of being a lost century. Even personally I can’t guarantee my data will be with me 20 years from now even though I back it up. If you care about a photo or document, print it and throw it it a box. As I get older I find more of an obsession with physical media from a preservation point of view. Because I know my books and pictures will be around 50 years from now. Digital files not so much.
This is a hot take
Even data from the Apollo missions was found to be either degrading (tapes) or the formats were forgotten and the systems that could read them were gone. They had to do research into rediscovering how to read the data and hunt around for the few antique systems remaining to read the tapes.
LOCKSS and KISS, though. Flash chips don’t last forever but are pretty durable, and so are optical media as long as they’re the right material. SSDs decay and HDDs fail, but for magnetic platter media even if the head or motor crashes there’s always the old magnetic microscope in a pinch. USB’s not going anywhere, and if you have four or five copies that you don’t completely neglect and don’t store in the same physical place, presumably you’ll have the chance to notice and take corrective measures if any of them start failing or are at risk.
I don’t actually know that an individual book or picture will still be around in 50 years; Fire, flooding, insects, acidic paper, low-quality ink maybe— Digital stuff’s fragile, but so is physical stuff. Stick it in the attic, and the heat’ll speed up any chemical reactions and probably make it cozier for insects; Stick it in the basement, and the condensation will get you mildew and rot. By contrast, having a flash drive accidentally survive a trip through a washer and dryer is a pretty common occurrence, and I’ve yet to lose a drive even with that level of negligence. Material compatibility’s one of the very most basic parts of a set of very precise manufacturing techniques, tin whiskers seem pretty rare these days, the really scarily insidious stuff like hydrogen embrittlement is super improbable, and most biological forms of decay haven’t adapted to eating cured epoxy and monocrystalline silicon yet.
At least I sorta know how a flash cell or hard drive platter is meant to be structured; Who knows what weird organic reactions and unstable or slowly diffusing molecules are happening in the pile of chemical pigments on a sheet of likely-acidic bleached cellulose and cheap ink or toner, and whether it will still be legible to human eyes in however many years? Plus, a printed photo or document starts fading the very instant it’s created, and it gets a little worse every time you touch it with sweaty human hands or look at it while exhaling moist human breath and corrosive enzymatic saliva droplets under a white LED lamp or G-type star shooting out ionizing UV rays. Digital failures tend to be catastrophic, but at least up until the moment it fails, you can make sure that it is the exact same picture or text— And you can make many, many copies very cheaply, all of them very physically durable compared to paper, and know that they are all the exact same picture and text.
That said, I absolutely agree with your overall assessment that most of the information in the early 21st century, including most of the public Internet/WWW, most likely either will be or already is… Maybe not technically lost, per se, given how much caching and saving happens on private clients, but certainly rendered inaccessible.
Ideally I’d really love to see a return of microfiche, actually, using modern polymers and metallization. I’ve been meaning to look into that for a while now. At a reasonable scale for optical viewing, you could fit… much, much more content than you might expect, and do it several times over, in an entirely reasonable number of pages. Your comment actually spurred me to finally think of a practical way of printing that— for years before, I’d been trying to idly figure out a process based on photomasks and nanoparticles suspended in resin, which had always felt like a very messy and tricky idea, but I just thought of another idea– So thanks for providing some inspiration there.
deleted by creator
There is a will but there aren’t enough people with enough brain power to actually do the steps needed. Should it endure? I don’t know, maybe the last few decades should be forgotten.
deleted by creator
Crazy how a single event sometimes reminds people of bigger problems, huh?
Digital media, where we store basically everything we care about, is hugely, hugely volatile, unreliable, and fragile. But you never notice it until you’re reminded of it, and then you really notice it. This story reminded people of it.
The reminder to stay grounded is probably also healthy, but I do think you’re missing the point of this comment thread.
Mistakes are usually the stuff that should be remembered
Fair
The Apollo mission data and BBC TV recordings weren’t considered important enough at the time to preserve them, it wasn’t until decades later that people realized they were but by then the BBC had destroyed or overwritten much of them and NASA had forgotten how to read much of the data. Then there was the notorious loss of many master recordings by great artists in a fire because the company was just too cheap and lazy to store them properly.
PCM, ASCII, and straight RGBA bitmap encodings aren’t going anywhere. By extension, derived formats like WAV, UTF-8, and word processor files and webpage HTML are mostly fine too. The formats are structurally simple enough that even if the associated file extensions were somehow to be forgotten, all you’d need to do to invent them again is hand the file to a bored nerd over the weekend.
I think you kinda got the BBC and NASA problems backwards. The BBC’s had a couple of prominent incidents where digital “preservation” that was supposed to be eternal couldn’t even be opened anymore after a couple of years, like their Domesday Book/Project application thingy. They’ve also lost a bunch of old shows, like early Dr. Who episodes, I think. NASA didn’t just forget how to read the Apollo tapes; they overwrote them to reuse the tapes, as was their standard practice at the time. The original signal and tapes were very HD (or analog), but most of the videos we have today are from the TV camera that they pointed at their own TV screen last-minute when they realized they didn’t have an adapter for broadcast— The equivalent of a grainy cell phone photo of a screenshot, basically.
The BBC and NASA incidents happened in an era before computers were a ubiquitous commodity product. So, everyone and their cat was basically inventing their own obscure single-implemention proprietary file formats at that time. Nowadays we have established technical standards, as well as formats that have already sorta stood the test of time based on their utility and simplicity— and millions of people who already know how to read them— so that particular vector for bitrot isn’t really as much of an issue anymore.
…That said, I think I sorta missed your point. What you’re really saying is that stewardship of digital records is much trickier and riskier than stewardship of physical records— and that results in stuff being lost. And that is absolutely true.
Many of the most culturally significant and creatively valuable works of literature, film, painting, and other forms of art and even science and engineering throughout history were widely panned, ridiculed, or simply unknown by their contemporaries.
“Will to maintain” comes from human people. And frankly, we’re vacillating, self-destructive, pleasure-seeking idiots with furthermore awfully finite practical limitations about what we can and want to have the will to do.
Lack of human eyes on something at a particular moment in time does not at all imply lack of its innate value. It doesn’t even guarantee lack of human eyes on it at a later date. Imagine if Da Vinci’s private notebooks, or Lovelace’s programs, or Van Gogh’s paintings, or Fermat’s humblebrag, or Melville’s writing and Dickinson’s poetry, or even Anne Frank’s diary, were saved on MicroSD instead of as physical hardcopy.
“Everything dies unless significantly effort and resources are constantly expended to maintain it” … Is not a world that I think I would prefer to live in.
…It would be an empty world and yet a violent world, filled with only the most trite and self-serving patterns designed and evolved to extract a profit in the briefest moment before they vanish. Even the stuff that does get “maintained” would have no fixed form, no verifiable truth, and no shared cultural experience or memory and heritage— Think of the streaming services that edit a film after it’s already been released, and imagine doing that to something like, E.G., Citizen Kane. Many of the things that we take for granted today— and probably most of our sense of cultural permanence, heritage, continuity, and memory— Might not really be possible in such a world.
And frankly, that’s going to bleed over into our politics too. Focusing exclusively on the single instant of the present because you no longer really have access to the past is a great way to make lots of selfish, impulsive decisions with disastrous consequence. It might even cause some degree of technological and cultural stagnation, as due to both psychological preferences and economic inequities, “will to maintain” is probably going to be heavily biased towards things that are already part of the status quo.
Physical media is at least somewhat inherently stable without constant maintenance on culturally relevant timescales, but digital media isn’t. At the same time that digital information technologies allow vastly greater quantities of valuable information to be assembled and disseminated, they also provide no practical way of passively preserving basically any of it. The invention of writing is fundamental to civilization, but the way it works is changing. Compared to the previous balance of capacities, we are now culturally capable of experiencing much, much more than we were before, but guaranteed to remember basically none of it— This is dangerous, or at least concerning and troubling.
Life is the perpetual fight against entropy, in all its forms. Meaningful information requires a lot of effort, work, insight, and feeling to put into words and pictures. Letting it be forgotten means forgetting who and what and why you are— Just because you can’t spare the CPUs and the technicians to keep its server up.
deleted by creator
This comment thread wasn’t about that one videographer. It was about backup techniques and data preservation practices in general, and worrying trends in that direction.
What is the advantage of using this over an USB to SATA adapter?
You don’t have to deal with using a USB to SATA adapter and the drive has a built in enclosure so you can just shove it into a bag or pocket
It’s extremely hard to make, but I was hoping there’s a review website with 100% real user review so this kind of issue can be discovered more easily
Got my gf a 2TB version. She also lost most of her files after 1 month of usage. She uses MacOS. But it’s probably to some degree of personal failure. So not sure if this is relevant.
So they just had this one drive fail and they decided to make a big news article about it? Hardware fails sometimes. Just RMA the thing and shut the fuck up about it. Go build a gaming PC.
deleted by creator
Single point of failure… surprised pikachu?
The use case for these drives is always a single point of failure. It’s camera footage out in the field. You have to get it home before you have proper storage.
Do you have a dashcam for your vehicle? Do you have backups before you get the footage home?
“I put 3TB of irreplaceable data on a single drive, and want to blame anyone but myself for my data loss”
Go away with this garbage.
I personally have a NAS with 12TB striped over 3 drives, I sure wouldn’t blame WD if one drive failed and I lost everything.
E: this whole comment section is why tech illiterate people shouldn’t really comment on hardware failures like this. The only fact that is know is that the verge faced 2 drive failures and lost 3TB of data due to a lack of safe data storage practices. If they were tech literate they wouldn’t have lost any data.
The verge did not confirm the mode of failure, and therefore the second failure could’ve been completely unrelated to the firmware issue. Nobody knows anything, other than the verge needs to educate themselves on how to properly store irreplaceable data.
Hence why RAID5 is so popular!
Correct. My next build will be redundant but given that my truenas pool is only storing movies, shows, music and porn, I don’t much care if I lose the contents due to a drive failure
The claim here seems to be that the product has an unusual failure rate, the manufacturer has acknowledged the original problem and released a fix, and it does not appear to be fixed. I don’t read it as a sob story about some reporter’s lost data.
Given the verges track record on tech reporting, i wouldn’t put faith in their journalistic integrity of a hit piece unless they show a bit more than “look, i lost a drive after they said they fixed the issue. They’re lying!”
Oh, okay, they just lost 3TB 😋
Fiew … it could have been 4.
deleted by creator
I purchased a 2TB one of these SanDisk “extreme portable” drives in 2018, and 2 more 2TB drives in 2019. Purchased each one roughly 6 months apart. Knock on wood…so far no problems at all with any of the 3. But, drives do often fail (I’ve had several fail over the years). One general rule of thumb I have when shopping for drives is I never buy the model with the highest storage capacity for the product line. It’s just a dumb superstition I have, but it seems like the higher capacity ones (like 3TB and above) are the ones that have failed on me in the past.