Core#

CLP core is the low-level component that performs compression, decompression, and search.

Requirements#

  • We have built and tested CLP on the OSes listed below.

    • If you have trouble building for another OS, file an issue, and we may be able to help.

  • A recent compiler that fully supports C++20 features such as

    • std::span

  • CMake >= 3.23.0 and < 4.0.0

  • Task 3.44.0

  • uv >= 0.8

To build, we require some source dependencies, packages from package managers, and libraries built from source.

Set up#

To initialize the project, run:

tools/scripts/deps-download/init.sh

Source Dependencies#

To get all dependencies required by CLP core, run the deps:core task:

task deps:core

The task will download, build, and install (within the build directory) the following libraries:

Library

Version/commit

abseil-cpp

20250512.0

ANTLR

v4.13.2

Boost

v1.87.0

Catch2

v3.8.0

date

v3.0.1

fmt

v11.2.0

liblzma

v5.8.1

log-surgeon

a722d07

lz4

v1.10.0

microsoft.gsl

v4.0.0

mongo-cxx-driver

r4.1.1

msgpack-cxx

v7.0.0

nlohmann_json

v3.11.3

simdjson

v3.13.0

spdlog

v1.15.3

SQLite3

v3.36.0

utfcpp

v4.0.6

yaml-cpp

v0.7.0

yscope-log-viewer

3abe4cc

ystdlib-cpp

9ed78cd

zlib

v1.3.1

zstd

v1.5.7

Environment#

A handful of packages and libraries are required to build CLP. There are two options to use them:

  • Install them on your machine and build CLP natively

  • Build CLP within a prebuilt docker container that contains the libraries; However, this won’t work if you need additional libraries that aren’t already in the container.

Native Environment#

See the relevant README for your OS:

Want to build natively on an OS not listed here? You can file a feature request.

Docker Environment#

You can use these commands to start a container in which you can build and run CLP:

# Make sure to change /path/to/clp/components/core and /path/to/my/logs below
docker run --rm -it \
  --name 'clp-build-env' \
  -u$(id -u):$(id -g) \
  -v$(readlink -f /path/to/clp/components/core):/mnt/clp \
  -v$(readlink -f /path/to/my/logs):/mnt/logs \
  ghcr.io/y-scope/clp/clp-core-dependencies-x86-ubuntu-jammy:main \
  /bin/bash -l

cd /mnt/clp

Make sure to change /path/to/clp/components/core and /path/to/my/logs to the relevant paths on your machine.

Build#

  • Configure the cmake project:

    mkdir build
cd build
cmake ../

  • Build:

    make -j

Test#

task tests:integration:core