ContentSync

SocialStack sites are built for high throughput out of the box, however, there always comes a scale where you need to expand on to multiple servers. You need to measure this for your site, but that scale tends to be around 5000 simultaneous users. Beyond this value, NGINX and Linux start to struggle with file pointers and the database connection pool starts getting too busy. Usually the step from 1 to 2 servers requires large amounts of engineering to ensure everything works together correctly. SocialStack has built in clustering mechanics, ContentSync, which helps servers work together on most things automatically by default.

How ContentSync works

Servers in a SocialStack site cluster register themselves with an entry in your database, as an entity called ClusteredServer. This entity tracks the servers current public and private IP addresses to allow servers to communicate directly with each other via a mesh network, over a datacenter private LAN whenever possible. Depending on the host(s) you're on, private VLANs can extend beyond a particular datacenter meaning the private IPs of your servers are reachable from any of your servers worldwide.

Once a server is registered as a ClusteredServer and has started communicating with other servers, they talk about 2 main things: the status of network rooms and cached entities. Network rooms provide an abstract mechanism for messaging a group of people (or a single person) who can be connected to any combination of servers, including one user connected to multiple servers. Cached entities, when edited on a server, triggers a message to all other present servers to tell them about the update. This allows cached objects to completely avoid hitting the database except for the server which triggered the update message in the first place.

AWS Elastic Load Balancing

Because SocialStack servers just need to exist and point at your database to start taking load and working together, SocialStack naturally works well with ELB. There is just one very important note: When ELB destroys and recreates servers, you need to make sure your hostnames are based on reused values. For example, if you have 3 servers in your ELB pool, their hostnames are always 1, 2 and 3. If 2 dies, the new replacement node gains hostname 2. This avoids your ClusteredServer table growing with hundreds of entries of dead servers. That is mostly only problematic for the block ID assignment mentioned below as it results in large chunks of your ID space becoming permanently unusable otherwise.

ID Assignment

SocialStack has multiple methods for assigning an ID to a new object.

Database assignment

The default for most entities is to use the method provided by the database: the latest database insertion ID. This has two problems:

• Not all database engines support it. MySQL is just the default - SocialStack doesn't actually rely on MySQL exclusively.
• It requires a centralised, singular counter. At high throughputs, keeping IDs from colliding becomes a bottleneck.

As a result, alternate methods are also present for the purpose of solving these problems. Most sites however won't actually encounter this as an issue.

Block assignment

The high throughput ID assignment mode, also activated when an entity is set to cache, is where each server in a cluster has exclusive ownership of a block of IDs. These blocks are spaced very far apart as they are formed by combining the ClusteredServer ID with the latest rolling number for that particular server.

If you are seeing very large IDs appear in your database, it is because there are multiple entries in your ClusteredServer table (site_clusteredserver) and block assignment is active. If your site is not live yet, or there is only 1 server in your live cluster, you can truncate this table and restart the server to have it assign lower IDs if you would like it to do so.

Low frequency block assignment

In this mode, clustered servers assume the object will be created at very low frequency and will self-assign IDs but will do it by discovering the latest assigned ID and adding 1. This always creates compact linear IDs and is similar to database assignment, except it does not rely on a central counter.

An example of an entity that uses low frequency block assignment is locales. Locales are edited rarely so they are cached meaning block assignment is active, but they also benefit greatly from having compact close together IDs due to the way how many SocialsStack features internally optimise localised sets. So, by setting it to low frequency block assignment the locales will end up tightly next to each other in the ID space - even if the database increment value is incorrect.

Because of this methods simplcitity ("latest plus one") - it is vulnerable to ID collisions if the object is actually created at a high frequency and the servers knowledge of the latest object is simply fractionally out of date.