Load Balancer Rust

Simple high-performance TCP-level load balancer / reverse proxy made in Rust

Why?

Because sometimes you just need a very simple tool with minimal configuration and that just works.

Also because I just wanted to try using Rust and this was fun to do.

Usage

A hosts file is required in the same directory from where you're calling the program. Should contain all servers' [HOSTNAME]:[PORT] on every new line.

Example hosts file content:

localhost:5000
127.0.0.1:5001
domain.com:80

Running the program: (will listen on port 7777)

./load-balancer-rust 7777

Add --verbose (or -v) to log every connection:

./load-balancer-rust 7777 --verbose

One worker thread is started per available CPU core.

Balancing algorithms

As of right now, only Round Robin is implemented. Every time connecting to a server fails or times out, the server is marked as unavailable and is avoided for some time. To avoid losing time on constantly trying to connect clients to an offline server.

If connecting to a server fails, the client is moved to the next server. A client that can't be connected to any server within 4 seconds is disconnected. Once a client is connected, it stays with that server until either side closes the connection. Connections with no activity for 10 minutes are closed.

Issues

Not yet fully optimized for Windows. Some weird behavior causing slower response times than on Linux.

Performance testing

Some load testing was done using the k6 tool to get an idea of relative performance. All testing was done on a system running Ubuntu 20.04.

The performed test was defined as:

import http from 'k6/http';
import { sleep } from 'k6';

export let options = {
  stages: [
    { duration: '20s', target: 1000 },  // slowly ramp-up traffic from 1 to 1000 users over 20 seconds
    { duration: '1m', target: 1000 },   // remain at 1000 users for 1 minute
    { duration: '20s', target: 0 },     // slowly ramp-down to 0 users
  ]
};

const BASE_URL = 'http://localhost:7777';

export default () => {
  let res = http.get(`${BASE_URL}/test_endpoint`);
  sleep(1);
};

Reference

The test was first ran directly against a local web server to get a reference point:

Average response time was around 0.25ms.

nginx 1.18.0

I then set up a reverse proxy on nginx like so:

server {
  listen 6666;
  listen [::]:6666;
  
  location / {
    proxy_pass http://localhost:5000
  }
}

And ran the test against nginx and got the following results:

Average response time was around 0.55ms. (an overhead of about 0.30ms)