LMDB as a Seq Storage Engine
I’ve run Linux for all kinds of tasks over the years, but Seq has been Windows-oriented since its inception. Making Seq run well on Windows was a very early goal, and the original value proposition still holds today: double-click an MSI and have a log server collecting events in under 5 minutes.
In the last couple of years there has been a growing interest in running Seq on Linux - no great surprise if you’ve been following the development of .NET Core. Now seems like a natural time to start bringing our plans for Linux support into motion.
We don’t want to go dark for months at a time working on a port, though, and I’m also wary of “big bang” style releases. Instead, we’re going to roll out Linux-compatible/.NET Standard components in Seq over the next few point releases, until we’re ready to flick the final switch.
Aside from the build, packaging, and configuration work, there are four main pieces that need to be developed before Seq-on-Linux will be a reality:
- Event Store — Seq uses ESENT for low-level storage on Windows; since ESENT is not available for Linux, an alternative storage engine needs to be integrated
- Document Store — similarly, ESENT is used by Seq’s metadata store (API keys, signals, retention policies, users…); this is a completely separate system that also needs a Linux-friendly alternative
- App Host — plug-in apps are hosted using
AppDomain
; on .NET Core, these will be hosted out-of-process instead - Web Server — last, but not least, there’s work to be done supporting Kestrel/nginx in addition to Windows’ http.sys web server
The first and largest of these is rolling out in Seq 4.1, which now supports pluggable storage engines, and includes LMDB as a selectable alternative in addition to ESENT.
LMDB is a low-level key value store which, thanks to Lightning.NET, is very easy to get running with the .NET Framework and .NET Core on Windows and Linux.
I don’t expect many customers will enable LMDB on Windows in production, but for those with an interest in seeing Seq on Linux (including ourselves), putting Seq + LMDB through its paces in the real world is a useful step along the path to a fully-supported Linux deployment option.
To enable it, remembering this is an early preview that may do anything up to and including eating your laundry :-), download the 4.1 preview build and run the following from an administrative command prompt on the Seq server:
seq config -k storage.preferredEngine -v LMDB
seq restart
Both ESENT and LMDB can be used within the same Seq instance: ESENT can be re-selected by specifying ESENT
rather than LMDB
in the above command. Changes won’t take effect until a new 7-day extent is created, and once an extent is initialized it will continue using the same storage engine regardless of the current preference.
What differences will you notice? None, hopefully! The familiar filtering and querying capabilities of Seq are implemented by Seq itself, with the storage engine providing only key-value persistence behind the scenes.
If you’re able to take Seq + LMDB for a spin, I’d love to hear about your experiences, and appreciate any and all feedback. The Seq issue tracker is the best place to post bug reports, if you spot anything amiss. Happy logging!