At about 11:20AM EST Thursday morning, we found ourselves under attack by an unknown hacker. He deleted the entirety of the forum's post database, deleted the front page news entries, and proceeded to slowly edit the main page to be branded as a site called "REALLY Sonic." The page that replaced the site's main index can be seen here; one of our various retaliation pages can be found here.
After several hours of us deliberating in the staff IRC channel sifting through Apache logs, FTP access logs, and various other logs full of raw numbers and timestamps, we eventually found our culprit—a member by the name of Shibunoa. In addition to his IP being found in the access logs for the server, his useragent—and his apparent use of NetBSD—also helped concrete his involvement in the attack. We don't know if anyone else was ever involved, but it can be safely assumed that the actual attack was his responsibility.
Once we found out how he got in, we worked on fixing the exploit. This was drx's job, as the vulnerability was a result of the nature of his "Sonic Dev FTP" service, in addition to Apache's apparent sentient access over the files on this server. He changed the password to the FTP, but he also made a small oversight—since Shibunoa also knew the URL for the HTTP section of the FTP, he was also able to get the new password listed on this page. While the FTP has since been deleted, the damage was soon to hit us harder than it did the first time.
After restoring backups of all of our lost data, we got hit a second time at the same exact time of day as the first. This time, though, we weren't as lucky—not only did he delete the SQL for the forums and the wiki, but he also wiped the images directory, taking literally thousands of files hosted on our wiki with it. Normally this wouldn't be a problem—after all, we had been in the process of making more backups for such an occasion—but Scarred Sun hadn't finished making a full backup, and only had 1,000 or so of the 10,000+ files that were originally in the directory. In addition, the uploads directory on the forum suffered the same fate. As of now, we've managed to re-obtain about 1,500 of the lost files.
In order to combat this potential staggering loss, GerbilSoft stepped up to the plate and downloaded an image of the entire CulTNET HDD, running several diagnostic tools that would hopefully be able to scan the drive for any deleted files that might have still existed on the disc. After hours upon hours of downloading, transferring, and analyzing the disc image, however, it appeared that none of the deleted files existed on the drive any longer. Despite this, however, GerbilSoft's willingness, effort, and expertise have culminated in him becoming the newest Sonic Retro administrator.
Here's the bright side to all this—while the WordPress posts no longer exist outside of the realms of Google cache, we have lost absolutely no forum posts or wiki pages. In terms of textual data, everything is still completely intact and will remain so.
Members are encouraged to simply re-upload any avatars or photos that were once present in the uploads directory on the forum, and they are also encouraged—with an ingenious method that Gerbil himself devised—to scan their hard drives for any files that may have once been present on the wiki. More information will be available in a separate announcements thread soon as to how exactly you can help the wiki restoration effort.
Here are the people you should thank for helping get this place back online:
- Xkeeper, who worked his ass off from the start to help combat the hack as it happened and sifted through logs upon logs to help find the culprit.
- drx, who provided a ton of insight and used the access he had to help us figure out what happened and how it happened.
- Saz, who came in to save the day and deliver the exact logs we needed to ultimately identify Shibunoa as the hacker
- Scarred Sun, who had thankfully just made full SQL backups the night before for both the forum and wiki, allowing us to come out completely unscathed when it comes to forum and wiki text.
- nineko, who did his absolute best to help us with both his technical and legal expertise, and whose diligent work on the wiki—and the upcoming restore effort—in both this situation and past situations, has been extremely invaluable.
- GerbilSoft, who worked his ass off—and still is—to help us keep everything as intact as possible, and for being more than willing to help provide technical advisory when we need it.
- GeneHF, for talking as much shit as ever and keeping our spirits up.
- Myself, for beating the shit out of people in #retro and trying to explore as many possibilities as possible in working this situation out. I don't like tooting my own horn, though, so I'll let the others speak for me if they want...
And for Shibunoa... well, we only have one thing to say to you:
Keep it classy, Retro!