This is a joke, I didn’t really lock myself out
good reason to take a day out, will tell it to my boss.
No connection, no hackers.
Happened to me once. Had a little Pi at my parent’s house and that was a nice excuse to visit them.
Except when you get there and don’t want to talk or do all the meeting and greeting until you know the server still works.
deleted by creator
even worse. I regularly have to get up out of my chair and go down 2 stairs.
Also this took a while to find, but : https://sourceforge.net/p/shorewall/svn/HEAD/tree/branches/4.2/Samples/one-interface/shorewall.conf
ADMINISABSENTMINDED=YesIs an actual setting in the config for the (now apparently unmaintained) Shorewall Firewall software/tool for linux.
If I remember correctly, it always checks on firewall rule changes if there is an active connection on port 22, and adds a special rule at the end to maintain that connection.
They don’t build them like they used to anymore.
They don’t build them like they used to anymore.
Well if we did, the way it works would be by telling a chatbot to enable ssh on port 22 at the end.
Real servers have lights out management and management networks.
I’d rather plug in a screen with VGA than deal with HPE iLO 4
Serial terminal servers (sometimes called terminal console servers) are a thing for a reason.
To be honest, HPE iLO 6 isn’t too bad, if you’re using the GUI. It’s the API that remains really broken in many places.
Networking noob here; what, pray tell, is HPE iLO4… or do I want to even know?
Edit: Never mind. Found it. HP… shudders
“In December 2021 Iranian researchers at Amnpardaz security firm have discovered rootkits in HPE’s iLO (Integrated Lights-Out) management modules.”
Because of course lol
Sounds like an issue draling with .NET or JRC console.
Are you on the nosz up to date firmware?I remember there being the option of using HTML or a Java applet, I chose the former
If you have the HTML5 option you should be on a pretty recent firmware.
Interesting that you’d prefer going (literally) analog connection rather than over the IPMI.
The latest version of iLO4 is from 2023
You know, I wanted to say “Bet!” and proove your wrong as I couldnt believe they never went past 2023 for the firmware.
Turns out that was the latest.But I do know they have more recent firmware uploads for the UEFI than 2023. ^(A year younger but no less nore recent/s)
What’s really fun is hearing “oh shit” from the UPS maintenance tech followed by darkness and silence.
Console
Fuck, that is really good wordplay.
Most secure box is the one that does nothing.
Don’t practically all commercial hosting providers provide remote console access?
This seems a combo of an extremely newb mistake in an extremely unusual scenario - worthy of Gru I guess.
Physical, on premises servers are still a thing.
They should have a remote console like Dell RAC or HP iLO
Could be they were configuring the actual network firewall and got a couple of rules out of order so they blocked all of their out of band access
Yes, I also used to run an “on premise” server - in my kitchen, not 500km away. I sometimes might need to admin it remotely, but never critical setup work.
And the meme makes it sound like they have to drive there specifically to fix it, like nobody is actually living nearby.I mean it’s a pretty realistic scenario. I happened to be the unlikely remote hands for the company I work for just a few weeks ago.
Company: an industrial cleaning company with about 1500 AD users and about 8000 employees, historically had 2 corporate offices, currently has three as it’s transitioning one corporate office across the country
Server and mistake in question: old admin who’s no longer with the company setup the ESXI 6.0 cluster in the server room at the office without documenting the root password to access it. This cluster happens to host the companies critical services including AD so being unable to access the host has been blocking the office migration. Old admin had also not fixed the ESXI backups which have been broken for over 3 years so no backups to restore from. Also the out of band access to the servers was never correctly setup
I happening to be close to this office and having IT experience was poked to go in and with physical access to modify the shadow file and set the root password to be blank. Had I not been available they would have had to fly someone in from the office 2000 miles away or hire a very expensive local contractor to come in after hours to do the same thing
Well, I have my server running in my parents basement, because they have fiber, and I don’t.
It’s not quite a 500km drive, but still a long enough distance for this scenario to be a major inconvenience.
But since I have wireguard running on their router though this specific scenario is not something that could happen to me
Wireguard is a VPN protocol, so you are able to tunnel into their router to…do what exactly?
It let’s me remote into their LAN, thus bypassing the firewall
Please forgive the ignorance here. What are you trying to do? I thought you were trying to reboot an offline server. I’m probably just confused!
Well, the original post (as in the image) is about locking yourself out of a remote server by changing a firewall rule, thus needing to drive to the server to access it locally.
By using wireguard to tunnel into the router, you can remotely enter the LAN, thus bypassing the firewall, as if you were accessing the server locally.
Yeah, all the ones I’ve used had remote access
I’ll always be grateful for the firewalls like OpenWRT that will automatically revert any changes if you don’t log back in after a few minutes (at least on the web interface). I’m not proud of how many times that’s saved me.
I try to remember to always open two SSH connections when altering iptables or the ssh config - just in case
Does it actually happen to people? All servers I worked with both had a back door (or two), and someone at the data centre (during work hours at least) you could contact in an emergency.
I guess some smaller companies might have simpler setups they self-host
Most data centers have some kind of service where you can request a KVM to be connected to the server. It’s not instant as an actual human has to do so but a lot sooner than another human driving long distance. I guess in this case, it’s a mid size company that is big enough to have multiple locations yet small enough to still manage to use on-premise infra instead of data centers.
iptables default DENY and flush the rules. Done by at least two people I know (then me) at the same company. Led to them moving the servers in-house and virtualizing some services to connect to the hypervisor. It does happen though.
This is precisely the problem that deploy-rs solves!
why is everything in rust now
It’s easy to write, easy to build, produces lightweight and fast executables, and the type system is great. Why not rust?
i feel that. Hetzner support has a special place in my heart
Rescue mode with networking, mount drive, make changes and reboot.
