This is an automated archive made by the Lemmit Bot.

The original was posted on /r/maliciouscompliance by /u/teddyzniggs on 2024-07-22 00:24:19+00:00.

TL;DR - a small e-comm I worked at had a horrible tech group that made a massive mistake. After getting roasted by everyone, they retaliated by blocking everyone’s ability to download/update any software/call any API’s without their approval. We retaliated by increasing API calls and library updates, costing them hours every day and getting them yelled at by the CEO for all reporting breaking.

I worked for a small e-comm company that had an odd dynamic between the CEO and the head of tech. There was a messy divorce between the two and the CEO and CTO and the CEO hired a new director of Tech who hired another guy as his head of BI and DE. Initially they seemed fine and they talked about all their plans about how they were going to fix all the old code that did make the systems and website slow and potentially crash. But as time went on, we all learned that these guys were literally just talk. They couldn’t deliver anything they promised, were very arrogant and took to taking 2-3 hour lunches at the bar and getting drunk. But because they kissed the CEO’s ass and because he hated the old CTO, he kept forgiving them, in spite of frequent protesting from the other teams in the company.

One day everything seemed to go awful. I lead the DS/Analytics team at that time and had a report that pulled in internal stats as well as API calls to google and adobe. Our live reporting showed that our conversion rate had fallen off and a quick check with the members of tech that were old hires of the old CTO and very reactive and competent confirmed that the website had suddenly gone down around 9:30am. We called director and the BI guy but nobody could get ahold of them. They rolled into work much later after the issue had been resolved. Everyone was pissed we could reach them but they claimed to just be heroes that quickly handled the issue which they stated definitively was a DDOS attack that they eventually managed to thwart. But soon the other tech guys that were there shared data that showed certain metrics were not acting consistently with a DDOS attack. After some pressure, we learned the head of BI was taking some randos through a tour of the offsite servers, saw a loose cable on the ground, picked it up, and just plugged it into some slot, which created a feedback loop or something that crashed the site. They had then gone to the bar and when they noticed the calls, they raced back, unplugged the cable and then rebooted the servers, causing everything to work again. They got ripped to shreds for their awful behavior, even the CEO piled on. The two jackasses never fully admitted this was the cause, claiming that it was a DDOS attack AND a coincidental and exactly timed feedback on the servers.

This is when they got petty. Claiming security was bad, they removed everyone’s ability download/upgrade/call any API without them signing off on literally everything. They hoped this would punish us all and initially we were really upset but the CEO was back to supporting their shenanigans. That’s when inspiration hit. Everyone impacted tried to do as any new downloads and updates we could possibly do. In particular, my team (using primarily R and Python at the time) began to require checks to upgrade every library in our code base, and created several scripts that had no real purpose other than to load lots of libraries, hit API’s and make plots. These tech guys didn’t know how to read either language and while they suspected shenanigans, they couldn’t prove it. It got so bad they would have to spend hours every morning with our team approaching every check with a password entry. The best came when they went into our code that WAS functional and commented out all API calls. The scripts failed to run and an important automated report couldn’t be updated, leading to the CEO being unable to update his presentation for the board. We showed him the commit that the BI guy had made and what he had done without consulting us. The ceo screamed at the guy for 30-40 minutes and was threatening to fire the guy. The arrogant jerk was reduced to tears and was crying and begging to keep his job. They immediately announced a new security patch and let us all download everything. I got a new gig shortly after but found out the two eventually were fired for incompetence and being drunk on the job. One of the awesome original tech guys is now running everything and as near as I know, there haven’t been any major problems since!