Maybe he left his shed lizard skin on top of the hard drive that caused the overheating?
Even a gamer knows that ssdd heat up but never to that level, lol.
What kind of cheap temu ssd does he have in his laptop?
He was saving money, you know, to be efficient.
60k of rows is nothing. Fuck, where do you find these “geniuses”?
I do data analysis for a living, I reach out to tech and complain if I can’t open a file with a million+ lines.
Tbf we don’t know how many columns there are /s
The IRS just switched columns and rows. So there’s 60k rows and 330 million columns /s
You’re kidding
Unless I’m misreading it which is possible it’s awfully late, he said he processed 60,000 rows didn’t find what he was looking for but his hard drive overheated on the full pass.
Discs don’t overheat because there was load. Even if he f***** up and didn’t index the data correctly (I assume it’s a relational database since he’s talking about rows) The disc isn’t just going to overheat because the job is big. It’s going to be lack of air flow or lack of heatsink.
I guarantee you he was running on an external NVMe, and one of those little shitty-ass Chinese enclosures. Or maybe one of those self immolating SanDisk enclosures. Hell, maybe he’s on a desktop and he slept a raw NVMe on his motherboard without a heatsink
There are times when you want a brilliant college student on your team, But you need seasoned professionals to help them through the things they’ve never seen before and never done before.
yes but also why say 60K when you could have literally said anything? I mean surely the fact that he thinks 60K rows a big number is already explaining alot lol.
It’s bait.
They probably an explanation tweet at the ready to make more sense of it. They just want enough 'hurr durr these idiot" comments before they reverse Uno card this with more context.
There are times when you want a brilliant college student on your team, But you need seasoned professionals to help them through the things they’ve never seen before and never done before.
Honestly, any sweet, white-haired old lady who keeps pictures of her dogs and grandkids on her desk who’s been doing data entry for 15 years could do circles around these clowns.
But she might also have the wisdom and perception to know we’re not supposed to be doing this “work” at all, which is why he recruits naive teenagers and college kids who are still emotionally immature to think that this is going to be their “destiny” or their opportunity to get into the big leagues of business.
I keep hearing things about these hires he has, I don’t think they’re naive, At least not as such. They seem to be more power hungry trust fund babies.
But yeah, people with a few years in them would be a moral liability in that line of work.
Yeah if you read more of these guys tweets they are clearly in politics. One message tried to claim trump loves kids (to be clear: in the abstract sense, not in the he definitely fucked kids on an island with Epstein sense). Then they tried to twist the words to say “why don’t you love kids”. It was clumsy like you’d expect from someone who is practically a teenager, but the core is an attempt to follow the usual right wing playbook.
Can’t be a relational database, Musk said the government doesn’t use SQL.
Lol he also said cybertrucks don’t suck ;)
He said many things.
Please remember that he is a genius. Only geniuses say a lot of things.
I rest my case.
grep would eat 60k lines without a problem too.
music theaters also have rows, and they run on sql so logic checks out.
Somehow I feel over clicking without understanding of the consequences sounds like something a techbro would do
You’re not supposed to place your laptop directly in the lap of your fur suit. Always leave an air gap for ventilation, smh.
This cannot be real, wtf. This is cartoon levels of ineptitude.
Or sabotage by someone heading out? Please let this be resistance sabotage they haven’t noticed yet.
You guys arent running your software off raspberry pi’s with sdcards from the gas station?
My allowance is 5$ a month!
I think my Pi could process 60k rows without overheating. And the poor thing is dangling behind my bookshelf from its power cord with a fine layer of dust coating every inch of it.
Why would you do this to that poor rPi?
Look, all I’m saying is give Pis a chance.
Either she knows something novel, where processing data using voice coils is somehow beneficial, or is someone who calls their computer a ‘hard drive’, which summarily negates any legitimacy of technical competence.
Or wrote the code using AI without checking what it exactly does.
The AI probably used bogosort or something equivalent
I will now attempt a full comment … brace your hard drives
Bro seriosuly fuck off my phone is overheating now. Thanks
I smell something, but it’s not overheating electronics.
I’ve processed over 5 million records on a laptop that’s almost 10 years old. it took two days to get my results.
there’s no way 60,000 records overheated ANYTHING.
Doesn’t actually say that 60k overheated his drive. He says that he ran a run on 60k, and that he couldn’t do the whole database due to overheating. Two unrelated statements except that 60k is the lower bound for what he could process.
Doesn’t mean he knows what he’s doing though, as pretty huge datasets are processable on quite modest hardware if you do it right.
that’s somehow worse.
a “data analyst” couldn’t cut up the work into a parallel processes and run them synchronously? what the actual fuck?
“sorry, I can only do 60k at a time.”
just fucking split them up into 6 parallel batch processes running 10k at a time. it’s fucking math, not rocket science. I’m not even an analyst and I could fucking do that much.
I don’t want to take away from the valid point of your comment, but rocket science is almost exclusively math
I’m a data engineer that processes 2 billion row 3000 column datasets every day, and I open shit in Excel with more than 60k rows. What the hell is this chick talking about?
Diming on her own stupidity without realizing it, like all of them do
Seems like a good excuse to someone who doesn’t know what they’re doing and needs an excuse because why they haven’t completed it yet?
The whole post is complete bs in multiple ways. So weird.
It sounds like Hollywood tech lingo. Like when you’re watching a movie or a TV show and the designated techy character starts just saying computer words that make no actual sense in the real world, but I guess in CSI: Idiottown the hard drives have severe overheating issues.
If you work for a boss that fundamentally misunderstands what you are doing, then misleading them into thinking you’re ‘hard at work, making decisions with consequences’ is the theatre you put up to keep the cash flowing.
It’s one of the fundamental flows of autocracy, people try and represent what you want them to
Some interesting facts about excel I learned the hard way.
- It only supports about a million or so rows
- It completely screws up numbers if the column is a number and the number is over 15 digits long.
Not really related to what you said, but I’m still sore about the bad data import that caused me days of work to clean up.
It completely screws up numbers if the column is a number and the number is over 15 digits long.
I work in insurance in Brazil, by standards of our regulatory body, claims numbers must be a string of 20 numbers (zfill(20) if needed). You can’t imagine the amount of times excel had fucked me up rounding down the claim numbers, this is one of the first things I teach to my interns and juniors when they’re working with the claims databases.
The row limitation seems, to me, like an actually-good thing. Excel is for data where you might conceivably scroll up and down looking at it and 1M is definitely beyond the ability of a human even to just skim looking for something different.
An older version of Excel could only handle 64k rows and I had a client who wanted large amounts of data in Excel format. “Oh sorry, it’s a Microsoft limitation,” I was thrilled to say. “I have no choice but to give you a useful summarization of the data instead of 800k rows (each 1000 columns wide) of raw data.”
Some time ago, I heard a story of CS and Econ professors having lunch together. The Econ professor was excited that Excel was going to release a version that blew out the 64k row limit. The CS professor nearly choked on his lunch.
Dependence on Excel has definitely caused bad papers to be published in the Econ space, and has had real world consequences. There was a paper years ago that stated that once a country’s debt gets above 120% of GDP, its economy goes into a death spiral. It was passed around as established fact by the sorts of politicians who justify austerity. Problem was, nobody could reproduce the results. Then an Econ undergrad asked the original author for their Excel spreadsheet, and they found a coding error in the formulas. Once corrected, the conclusion disappeared.
“I store my records on vinyl. You’ve probably never heard of them.”
60k rows of anything will be pulled into the file cache and do very little work on the drive. Possibly none after the first read.
You can put 60k rows in Excel 95.
Not if each row is pi!
When the only thing that is stopping kids from dismantling your government is an O(N^N) algorithm
Are you telling me there’s a difference between an inner and a cross join?
Cross join is obviously faster, I don’t even have to write “on”
my hard drive overheated
So, this means they either have a local copy on disk of whatever database they’re querying, or they’re dumping a remote db to disk at some point before/during/after their query, right?
Either way, I have just one question - why?
I’d much sooner assume that they’re just fucking stupid and talking out of their ass tbh.
Same as Elon when he confidently told off engineers during his takeover of Twitter or gestures broadly at the Mr. Dunning Kruger himself
Wonder if it’s an SQL DB
Elon probably hired confident right wingers whose parents bought and paid their way through prestigious schools. If he hired anyone truly skilled and knowledgeable, they’d call him out on his bullshit. So the people gutting government programs and passing around private data like candy are just confidently incorrect
Have you ever heard of case of overheating hard drives within the last decade?
Plus, 60k is nothing. One of our customers had a database that was over 3M records before it got some maintenance. No issue with overheating lol
I run queries throughout the day that can return 8 million+ rows easily. Granted, it takes few minutes to run, but it has never caused a single issue with overheating even on slim pc’s.
This makes no fucking sense. 60k rows would return in a flash even on shitty hardware. And if it taxes anything, it’s gonna be the ram or cpu- not the hard drive.
In my experience, the only time that I’ve taxed a drive when doing a database query is either when dumping it, or with SQLite’s vacuum, which copies the whole thing.
For a pretty simple search like OP seems to be doing, the indices should have taken care of basically all the heavy lifting.
I literally work with ~750,000 line exports on the daily on my little Lenovo workbook. It gets a little cranky, especially if I have a few of those big ones open, but I have yet to witness my hard drive melting down over it. I’m not doing anything special, and I have the exact same business-economy tier setup 95% of our business uses. While I’m doing this, that little champion is also driving 4 large monitors because I’m actual scum like that. Still no hardware meltdowns after 3 years, but I’ll admit the cat likes how warm it gets.
750k lines is just for the branch specific item preferences table for one of our smaller business streams, too - FORGET what our sales record tables would look like, let alone the whole database! And when we’re talking about the entirety of the social security database, which should contain at least one line each in a table somewhere for most of the hundreds of millions of people currently living in the US, PLUS any historical records for dead people??
Your hard drive melting after 60k lines, plus the attitude that 60k lines is a lot for a major database, speaks to GLARING IT incompetence.
Pretty sure I run updates or inserts that count over 60k fairly often. No overheats. Select queries sometimes way higher.
You’ve got it all wrong, in traditional computer terminology the “hard drive” is the box that sits under the desk that collects cat fluff and cigarette tar.
/s …?
I don’t think I’ve seen a brand new computer in the past decade that even had a mechanical hard drive at all unless it was purpose-built for storing multiple terabytes, and 60K rows wouldn’t even take multiple gigabytes.
Reminds me of those 90s ads about hackers making your pc explode.
Musk gonna roll up in a wheelchair, “the attempt on my life has left me ketamine addicted and all knowing and powerful.”
I have when a misconfigured spark job I was debugging was filling hard drives with tb of error logs and killing the drives.
That was a pretty weird edge case though, and I don’t think the drives were melting, plus this was closer to 10 years ago when SSD write lifetimes were crappy and we bought a bad batch of drives.
Even if it was local, a raspberry pi can handle a query that size.
Edit - honestly, it reeks of a knowledge level that calls the entire PC a “hard drive”.
Unless they actually mean the hard drive, and not the computer. I’ve definitely had a cheap enclosure overheat and drop out on me before when trying to seek the drive a bunch, although it’s more likely the enclosure’s own electronics overheating. Unless their query was rubbish, a simple database scan/search like that should be fast, and not demanding in the slightest. Doubly so if it’s dedicated, and not using some embedded thing like SQLite. A few dozen thousand queries should be basically nothing.
My one question would be “How?”
What the hell are you doing that your hard drives are overheating? How do you even know it’s overheating as I’m like 90% certain hard drives (except NVMe if we’re being liberal with the meaning of hard drive) don’t even have temperature sensors?
The only conclusion I can come to is that everything he’s saying is just bullshit.
They have temp sensors. But have never heard of a overheating drive.
Hard drives do get hot and need some cooling but not at 60k rows. Its either made up or their computer case is made of thermal cladding
You could query 60,000 rows on a low tier smart phone. Makes no sense at all.
Can we think of any device someone might have that would struggle with 60k? Certainly an ESP32 chip could handle it fine, so most IoT devices would work…
Unless the database was designed by someone who only knows of data as that robot from Star Trek, most would be absolutely fine with 60k rows. I wouldn’t be surprised if the machine they’re using caches that much in RAM alone.
Right? There’s no part of that xeet that makes any real sense coming from a “data engineer.”
Terrifying, really.
Imo if they can’t max out their harddrive for at least 24 hours without it breaking, their computer was already broken. They just didn’t know it yet.
Any reasonable SSD would just throttle if it was getting too hot, and I’ve never heard of a HDD overheating on its own, only if there’s some external heat sources, like running it in a 60°C room
Hard Drives might do it if the enclosure is poorly designed (no ventilation), but I can’t imagine a situation where it would overheat like that that quickly, even in a sealed box. 30k is nothing in database terms, and if their query was that heavy, it would bottleneck on the CPU, and barely heat the drive at all.
dude is 100% talking about ssds. NVME ones at that, he’s just stupid.
Why? Because they feel the need to have local copies of sensitive financial information because… You know… They are computer security experts.
Or they’re doing it on a Diamondmax 9.
From the same group that doesn’t understand joins and thinks nobody uses SQL this is hardly surprising .
Probably got an LLM running locally and asking it to get data which is then running 10 level deep sub queries to achieve what 2 inner joins would in a fraction of the time.
You’re giving this person a lot of credit. It’s probably all in the same table and this idiot is probably doing something like a for-loop over an integer range (the length of the table) where it pulls the entire table down every iteration of the loop, dumps it to a local file, and then uses plain text search or some really bad regex’s to find the data they’re looking for.
Considering that is nearly exactly some of the answers I’ve received during the technical part of interviews for jr data eng, you’re probably not far off.
Shit I’ve seen solutions done up that look like that, fighting the optimiser every step (amongst other things)
I think you’re still giving them too much credit with the for loop and regex and everything. I’m thinking they exported something to Excel, got 60k rows, then tried to add a lookup formula to them. Since you know, they don’t use SQL. I’ve done ridiculous things like that in Excel, and it can get so busy that it slows down your whole computer, which I can imagine someone could interpret as their “hard drive overheating”.
I have to admit I still have some legacy code that does that.
Then I found pandas. Life changed for the better.
Now I have lots if old code that I’ll update, “one day”.
However, even my old code, terrible as it is, does not overheat anything, and can process massively larger sets of data than 60,000 rows without any issue except poor efficiency.
They don’t understand joins? How…
