BARF database corruption?

Abacinator

Unholy Blasphemies
^ Yeah, limiting post count seems like a reasonable workaround.

The "thinking" thread has more than 50,000 posts--twice as many as the next biggest one I found--and is the only one with more than 32,767 (upper limit of signed 16-bit integer). Programming intuition suggests that's a possible problem.

Since impact is so limited, splitting that thread would make sense. Maybe also keep an eye on "Animated GIFs" in Pics & Vids with more than 25,000.

Isn't the Post Whore thread purged at some regular interval?
 

tzrider

Write Only User
Staff member
The error that we're seeing is sometimes associated with certain plugins (we don't use the one that is most commonly an issue) as well as certain nginx versions.

While I don't know for sure, I don't think those causes are going to be the problem in this instance.

Trimming the thread is a possibility if it comes to that, though it would undermine one of the original purposes of the thread; to be able to easily scan back over years to see what you (or someone else) was thinking at the time.
 

rodr

Well-known member
Sounds oddly like a problem with concurrent access. Big threads are more likely to have 2 or more people posting in them at the same time.

I say "oddly" because PHP setups typically run a web request to completion before starting another one. If some optimization was installed to defeat this, I would start by undoing whatever that was.

Have you reached out to vBulletin support to see what they say?
 

tzrider

Write Only User
Staff member
We have some unusual plug-ins that are probably over support's head and possibly related to the current problem. I'm working with our sysadmin to determine next steps.

We're definitely not ignoring this but the issue may persist for a little while.
 

W800

Noob
Ping shows 74.116.248.212.

We use Cloudflare so that we are not directly on the internet. Everything goes through Cloudflare before us, and the Cloudflare edge servers deliver most of our content. We also locked down our backend site management pages so that they require 2FA. Some days, we get literally hundreds of attempted intrusions. And this is for just a few small sites.

Another nice thing about Cloudflare is you can block (or force a CAPTCHA) based on country, browser, IP range, user agent, etc. . . For example, anyone visiting our sites using TOR is presumed to not be within our target markets, and gets a browser challenge. We also block most robots, since we aren't concerned about indexing.

The result of the above is that our server loads are minimal. Another thing that we do sometimes is run yet another CDN at AWS S3 for content that doesn't change a lot. This also makes pages load fast for a second reason - browsers will only open a number of simultaneous threads/connections to a single site. But if you load that single site from several locations, it will load faster.

ETA: we also get a second level of security. By running through Cloudflare, people don't actually know who hosts us. It's AWS - no big secret, but our attack surface is smaller.
 
Last edited:

rodr

Well-known member
BARF is a dynamic site, so CDN is not really applicable. Most interactions are specific to the logged-in user and will cause a database update of some sort.
 

tzrider

Write Only User
Staff member
While we haven't identified the cause, it isn't server load. A CDN wouldn't help the issues we're seeing.
 

W800

Noob
The sites we protect are dynamic as well. There's differing cache levels. For example, pages people see here are obviously dynamically generated. But the images, CSS and JavaScript are not. That stuff can be easily cached. I attached a pic of what can be cached. Everything except that last .php script can easily and automatically be cached at a CDN.

Even threads that haven't been updated can be cached. Cloudflare looks for the "last modified" header when it decides whether to send something from its cache or something from the origin.

It also changes images to WebP, and this can save some server load.

Although server load isn't the topic of this thread, my intuition is that the site is running at a high load, especially when a lot of users are here. This slows it down and messes up the UX.

Reason I bring up Cloudflare is that the basic functionality is free. There's also a $20 per month account that has even more features.

attachment.php
 

Attachments

  • cloudflare.png
    cloudflare.png
    78.6 KB · Views: 81

W800

Noob
Good info, thanks. Free is good!

Welcome!

Figured you might like that part.

:thumbup

The other nice thing is that nothing needs to change at the origin site. The way you enable Cloudflare is you just change your DNS settings to use Cloudflare's nameservers.

There's also some cool things you can do inside Cloudflare with the firewall they have. For example, you can set it so that only people from certain IPs are whitelisted to access certain pages (like your admin area). You can also implement zero trust rules using Cloudflare for Teams. That's also free!!!

And - if it doesn't work for your needs, just go back to your domain provider and revert your nameservers.

Disclaimer - I don't work for them. Just like the product.

ETA: there may be a couple of small changes on your end - see: https://support.cloudflare.com/hc/en-us/articles/202032590-Using-CloudFlare-with-vBulletin-forums
 
Top