IT workers, what's the wildest thing you discovered you had to fix on the job? #redditstories — Transcript

Full transcript

1. 0:00IT workers, what's the wildest thing you
2. 0:01discovered you had to fix on the job?
3. 0:02>> firm hired me to fix their crawling
4. 0:04network. First day, the IT director gave
5. 0:05me one rule: Do not touch the beige box
6. 0:07in server closet B, ever. I asked what
7. 0:09it did. He smiled like I'd asked
8. 0:10something stupid. It runs the firm,
9. 0:11powers the billing, the case files, all
10. 0:13of it. The guy who built it left years
11. 0:14ago and nobody understands it. We keep
12. 0:15the room cold and we don't ask
13. 0:16questions. Then he walked me to the
14. 0:17closet and showed me a dented beige
15. 0:19tower from the 90s humming under a
16. 0:20tangle of cables with masking tape that
17. 0:22read, "Do not shut down." A second AC
18. 0:23unit ran full blast just for that one
19. 0:25machine. The IT director's name was
20. 0:26Roland and Roland had built his entire
21. 0:28tenure career at that firm being the
22. 0:30only person brave enough to manage the
23. 0:31box. He got a bonus every year for
24. 0:33critical infrastructure maintenance. He
25. 0:34had a reserved parking spot. At the
26. 0:35holiday party, he'd say, "She's
27. 0:36temperamental, but I keep her happy."
28. 0:38Everyone treated him like a wizard
29. 0:39because nobody understood what he did.
30. 0:40Here's the thing, for 2 weeks I wasn't
31. 0:42allowed near the box, so I did the next
32. 0:43best thing. I traced the cables. Every
33. 0:44wire coming out of that beige tower ran
34. 0:46to a switch and from that switch nothing
35. 0:47important flowed anywhere. The billing
36. 0:49software, cloud hosted, had been for
37. 0:50years. The case files, on a separate
38. 0:52server two racks over that I was allowed
39. 0:53to touch. I put a network monitor on the
40. 0:55beige box and watched it for 3 days. It
41. 0:56sent and received almost nothing. A
42. 0:58trickle of traffic to one address. A
43. 0:59folder on the firm's own backup drive, a
44. 1:01folder full of Roland's personal movie
45. 1:02collection. The sacred box was a media
46. 1:04server. Roland had been streaming films
47. 1:05to his office for a decade and billing
48. 1:07the firm to keep his private Netflix
49. 1:08cold. I didn't say a word. I documented
50. 1:10everything. The traffic logs, the cable
51. 1:11map, the fact that the critical machine
52. 1:12touched zero critical systems. I
53. 1:14exported the movie folder's contents to
54. 1:15a clean report. Then I waited because
55. 1:17rushing it would just get me called a
56. 1:18liar by the guy with 10 years of trust.
57. 1:19My chance came at the quarterly IT
58. 1:21review. The one meeting where the
59. 1:22managing partner actually shows up.
60. 1:23Roland presented first, as always,
61. 1:25warning everyone in grave tones that the
62. 1:26beige box was aging and that he'd need a
63. 1:28bigger budget next year to keep it
64. 1:29stable. The partners nodded, he'd
65. 1:30trained them well. Then the managing
66. 1:31partner turned to me, "Anything to add
67. 1:33from the new hire?" "Just one thing," I
68. 1:34said, "I'd like to shut the beige box
69. 1:35down, right now, during this meeting."
70. 1:37The room went cold. Roland laughed too
71. 1:38loud, "Absolutely not. That machine runs
72. 1:40the firm. You take down billing, the
73. 1:41case files, everything. This is exactly
74. 1:43why we don't let people who don't
75. 1:44understand the system touch it." "Okay,"
76. 1:45I said, "then let's test it. If shutting
77. 1:47it off takes down anything the firm
78. 1:48needs, you're right and I'll apologize
79. 1:49to everyone here. If nothing happens, we
80. 1:51open it up and see what's actually on
81. 1:52it." I looked at the managing partner.
82. 1:53"10 minutes, worst case, we plug it back
83. 1:55in." The partner, who signed off on
84. 1:56Roland's bonus every year, wanted to
85. 1:57know why he'd been paying for something
86. 1:58that couldn't survive a 10-minute test.
87. 2:00He told me to do it. Roland's face
88. 2:01drained as I walked to closet B and
89. 2:03pulled the plug. We sat in silence.
90. 2:04Billing stayed up. The case system
91. 2:05stayed up. Email, phones, the document
92. 2:07drive, all fine. 5 minutes, 10. Nothing
93. 2:09so much as flickered. Then I connected
94. 2:10my laptop to the beige box and put it on
95. 2:12the projector. Since it's off anyway,
96. 2:13let's finally see the critical
97. 2:14infrastructure. I opened the drive.
98. 2:15Folder after folder filled the screen.
99. 2:17Not one case file, not one invoice, just
100. 2:19movies. Hundreds of them, sorted by
101. 2:20genre, plus a subfolder helpfully
102. 2:22labeled Roland watch later. I opened my
103. 2:23report beside it. 10 years of that
104. 2:25machine's traffic. Every two and from
105. 2:27his office and the maintenance line item
106. 2:28next to it. This is what the firm has
107. 2:29been cooling, powering, and paying a
108. 2:30bonus to protect, I said. It's a movie
109. 2:32server. It has never touched a single
110. 2:33thing this firm runs on. The managing
111. 2:34partner read the numbers on the screen.
112. 2:36The bonuses, the second AC unit, the
113. 2:37special budget requests added up over 10
114. 2:39years. Roland started stammering about
115. 2:40redundancy, about legacy systems, about
116. 2:42how I didn't understand the
117. 2:43architecture. The partner raised one
118. 2:44hand and Roland stopped like his own
119. 2:45machine had lost power. Roland cleaned
120. 2:47out his desk that afternoon. Security
121. 2:48carried the beige box out to the
122. 2:49dumpster behind him. The reserved
123. 2:50parking spot is mine now.