Invoking Eclair code

In this guide, we learn how to use the Eclair compiler to compile and run Eclair programs to executable code. This tutorial assumes that eclair is already installed on your machine.

In the commands on this page, $ECLAIR should be replaced with one of two options:

  1. eclair if you installed eclair directly on your system,
  2. docker run --rm eclair:latest -v $(pwd):/code eclair if you previously setup the eclair compiler via docker

Discovering compiler commands and options

You can use the -h or --help flag to get more information about how to use Eclair or one of it’s subcommands:

$ $ECLAIR -h
$ $ECLAIR compile -h

This can be a useful way to discover the different features of the Eclair compiler.

Compiling to LLVM

The compile subcommand can be used to compile Eclair programs to various output formats using the --emit flag. The default format is LLVM IR, but each of the internal representations during compilation can also be generated (sometimes useful for understanding how the program will evaluate).

Given the following example Eclair program:

@def edge(from: u32, to: u32) input.
@def reachable(start: u32, end: u32) output.

reachable(x, y) :-
  edge(x, y).

reachable(x, y) :-
  edge(x, z),
  reachable(z, y).

Then it can be compiled to LLVM IR with the following command:

$ $ECLAIR compile --emit llvm path.eclair > path.ll

Compiling to native executable code

After you have compiled your program to LLVM, you can use all of the tooling from the LLVM framework (such as clang, llc, opt, …).

For example, you can compile the generated LLVM file from the previous section to an object file using the next command:

$ clang -c -o path.o path.ll
# OR:
$ llc-14 -filetype=obj -o path.o path.ll

After that you can turn the object file into a static library archive:

$ ar rcs libpath.a path.o

The section about integrating with other languages will show how to link this library into a final executable.

Compiling to WebAssembly

In order to run Eclair code compiled to WebAssembly, we first need to compile a general-purpose allocator since none are provided in WASM. This is different compared to the previous section where we could automatically re-use malloc and free from the system-provided libc).

The code below shows how we can use walloc to use a WASM-specific allocator in our Eclair code:

# First we need to compile walloc.c
$ git clone https://github.com/wingo/walloc
$ cd walloc
$ clang --target=wasm32 -mbulk-memory -nostdlib -c -o walloc.o walloc.c
# Now we can compile the Eclair program and link it with walloc
$ $ECLAIR compile --target wasm32 path.eclair > path.ll
$ clang --target=wasm32 -mbulk-memory -nostdlib -c -o path.o path.ll
$ wasm-ld --no-entry --import-memory -o path.wasm path.o walloc.o

Note the use of the target flag to tell Eclair to generate WASM-specific code. After running these commands, a path.wasm file that can be imported and executed in Node.js or in a web browser.

Integrating with other languages

Eclair is designed to be easily invoked from other languages. The Eclair program runs in the same process as the other language (also referred to as the host language). Data (in- and output facts) are send back and forth between the languages via API calls.

The runtime API provided by Eclair provides all necessary functionality, but is quite low-level and error-prone to use by hand. For this reason, high-level language-specific bindings exist for the following languages to make integration with Eclair much easier:

  1. Rust
  2. Haskell
  3. Typescript and Javascript
  4. C

Missing bindings for your language? Let us know by submitting an issue on the Eclair repository.

Rust

Calling Eclair code from Rust requires the following libraries:

  1. eclair_bindings
  2. eclair_bindings_derive
  3. eclair-builder

You can add these dependencies with cargo:

$ cargo add eclair_bindings
$ cargo add eclair_bindings_derive
$ cargo add --build eclair-builder

Next, we need to add a custom build.rs that tells Rust how to link the Eclair code into a final executable. The eclair-builder crate gives us a convenient way to do just this:

extern crate eclair_builder;

use std::env;

fn main() {
    // NOTE: for now Eclair requires DATALOG_DIR to be set to
    // find the Datalog files used by the Eclair compiler.
    // Once the compiler is bootstrapped, this will no longer
    // be needed.
    let datalog_dir = env::var("DATALOG_DIR")
      .expect("'DATALOG_DIR' env var needs to be set!");

    // The next line of code tells Rust where to find each of
    // the tools required to build Eclair code.
    eclair_builder::Build::new()
        .eclair("eclair")
        .clang("clang-14")
        .datalog_dir(&datalog_dir)
        .file("src/analysis/path.eclair")
        .compile();
}

The code above tells cargo and rustc how to link with our Eclair code. Now that our Rust project is fully configured, we can write the code that runs our Eclair program using the eclair_bindings and eclair_bindings_derive crates:

extern crate eclair_bindings;
extern crate eclair_bindings_derive;

use eclair_bindings::*;
use eclair_bindings_derive::fact;

// Path is a struct used as a handle for the Eclair program.
struct Path;

// For each fact, we create another struct.
// A proc-macro generates all the necessary bindings.
#[fact(program = Path, direction = input, name = "edge")]
struct Edge(u32, u32);

// Named struct fields are also supported.
#[fact(program = Path, direction = output, name = "reachable")]
struct Reachable {
    start: u32,
    end: u32,
}

fn main() {
    // Start the Eclair runtime:
    let mut eclair = Program::new(Path);

    // Add some input facts:
    let edges = vec![Edge(1, 2), Edge(2, 3)];
    eclair.add_facts(edges.into_iter());
    eclair.add_fact(Edge(3, 4));

    // Calculate all results in Eclair:
    eclair.run();

    // Retrieve output facts:
    let results: Vec<Reachable> = eclair.get_facts().collect();

    // Process results ...
}

As you can see, the bindings provide a concise type-safe DSL that should prevent most logic errors while sending data back and forth between Rust and Eclair.

Haskell

The Haskell bindings can link with Eclair code that has been compiled to native code. The language bindings provide a lot of type safety given an accurate type-level description of the Datalog program.

First, you will need to update your cabal or hpack project file to add the eclair-haskell dependency and so that cabal detects the Eclair files. The following snippet shows how to do this for a project that makes use of hpack:

executables:
  eclair-example:
    source-dirs: src
    main: Main.hs
    # Add eclair-haskell as a dependency
    dependencies:
      - eclair-haskell
    # This assumes the library is stored under "cbits/"
    extra-lib-dirs: cbits
    # This will try to link with libpath.a
    extra-libraries: path

Once this is done, you can generate a static library archive from the Eclair code like previously shown in compiled to native code. Make sure that the resulting libpath.a is stored under the directory you mentioned in the extra-lib-dirs configuration.

Now that all the files are in place, you can use the following code to bind to Eclair and serialize data back and forth between Haskell and Eclair:

{-# LANGUAGE DataKinds #-}
{-# LANGUAGE DeriveGeneric #-}
{-# LANGUAGE DeriveAnyClass #-}
{-# LANGUAGE DerivingVia #-}
{-# LANGUAGE UndecidableInstances #-}
-- UndecidableInstances is only needed for the DerivingVia API

module Main ( main ) where

import qualified Language.Eclair as E
import GHC.Generics

data Edge
  = Edge Word32 Word32
  deriving (Generic)
  deriving anyclass E.Marshal
  deriving E.Fact
  via E.FactOptions Edge 'E.Input "edge"

data Reachable
  = Reachable Word32 Word32
  deriving (Show, Generic)
  deriving anyclass E.Marshal
  deriving E.Fact
  via E.FactOptions Reachable 'E.Output "reachable"

data Path = Path
  deriving E.Program
  via E.ProgramOptions Path '[Edge, Reachable]

main :: IO ()
main = do
  -- "withEclair" automatically starts and stops Eclair.
  -- It performs all necessary cleanup.
  results <- E.withEclair Path $ prog -> do
    E.addFacts prog [Edge 1 2, Edge 2 3]
    E.addFact prog $ Edge 4 5
    E.run prog
    E.getFacts prog
  process results
  where
    process :: [Reachable] -> IO ()
    process = traverse_ print

Typescript and Javascript

The Typescript and Javascript bindings are different compared to the other languages, since here we need to compile Eclair to WebAssembly. This makes it possible to run Eclair both in the browser and in Node.js.

To use Eclair in your project, add the following dependency:

$ npm install eclair-wasm-bindings

Next, we can use the language bindings to describe what the Eclair program looks like (which input and output facts). For the example that’s been mentioned previously on this page, it looks as follows:

import {
  withEclair,
  fact,
  program,
  U32,
  INPUT,
  OUTPUT,
} from 'eclair-wasm-bindings';

// Due to how WASM works, we need to provide Eclair enough memory
// to run. The amount you need to provide depends on how much data
// you will be processing with Eclair.
const memory = new WebAssembly.Memory({
  initial: 10,
  maximum: 100
});

// Fetch and compile the WASM program.
// This can be from anywhere on the internet!
const { instance: wasmInstance } = await WebAssembly.instantiateStreaming(
  fetch('/path/to/eclair_program.wasm'),
  { env: { memory } }
);

// Now start Eclair using "withEclair". This automatically takes
// care of resource cleanup as well.
const results = withEclair(wasmInstance, memory, (handle) => {
  // Next we define what the Eclair program looks like.
  // Important: This has to match *exactly* with how you defined it
  // in Eclair, otherwise you will get unexpected results!
  const edge = fact('edge', INPUT, [U32, U32]);
  const reachable = fact('reachable', OUTPUT, [U32, U32]);
  const path = program(handle, [edge, reachable]);

  // Now add facts to Eclair (LSP provides autocomplete!)
  path.edge.addFact([1, 2]);
  path.edge.addFacts([
    [2, 3],
    [3, 4],
  ]);

  // Let Eclair do the number crunching..
  path.run();

  // And finally you can get results back out!
  const reachableFacts = path.reachable.getFacts();

  // You can do anything with the results here..
  console.log(reachableFacts);

  // Or you can return the results so they can be used
  // outside this function!
  return reachableFacts;
});

Using Typescript in combination with these bindings is highly recommended because of the heavy use of types. In the example above, the LSP would auto-complete all the input and output fact names, as well as the available methods on them.

C

Because Eclair compiles down to LLVM and has a C-ABI compatible API, it is possible to call Eclair from C (or any language that supports C via a Foreign Function Interface (a.k.a. FFI)). In fact, this is how all the different language bindings support Eclair under the hood.

For C, there is no high-level API to use though. Only the low-level runtime API is available. Because of this, it is harder to use and easier to make mistakes. On the flip side, technically you could now also perform (potentially unsafe) tricks to save time sending data back and forth with Eclair (by avoiding copies, passing pointers around, …).

The code below is an example of how you could run the Eclair program directly from C:

#include <stdio.h>
#include <stdlib.h>
#include <stdbool.h>
#include <stdint.h>

// The Eclair runtime is represented by an opaque pointer
// to a program struct.
struct program;

// The low-level Eclair API is accessed via extern definitions
extern struct program* eclair_program_init();
extern void eclair_program_destroy(struct program*);
extern void eclair_program_run(struct program*);
extern void eclair_add_facts(
  struct program*,
  uint32_t fact_type,
  uint32_t* data,
  size_t fact_count
);
extern void eclair_add_fact(
  struct program*,
  uint32_t fact_type,
  uint32_t* data
);
extern uint32_t* eclair_get_facts(
  struct program*,
  uint32_t fact_type
);
extern uint32_t eclair_fact_count(
  struct program*,
  uint32_t fact_type
);
extern void eclair_free_buffer(uint32_t* data);
extern uint32_t eclair_encode_string(
  struct program*,
  uint32_t length,
  const char* str
);
extern struct symbol* eclair_decode_string(
  struct program*,
  uint32_t index
);

int main(int argc, char** argv)
{
    struct program* prog = eclair_program_init();

    // For each fact type, we need to retrieve the unique number
    // that Eclair uses to represent it.
    uint32_t edge_fact_type = eclair_encode_string(
      prog, 4, "edge"
    );
    uint32_t reachable_fact_type = eclair_encode_string(
      prog, 9, "reachable"
    );

    // Adding facts:

    // Facts are stored in a flat, contiguous array.
    // *NO* checks are made regarding size and fact count!
    uint32_t data[] = {
        1, 2, // edge(1,2)
        2, 3  // edge(2,3)
    };
    eclair_add_facts(prog, edge_fact_type, data, 2);

    // Calculating results with Eclair:
    eclair_program_run(prog);

    // Retrieving output facts:
    uint32_t fact_count = eclair_fact_count(
      prog, reachable_fact_type
    );

    // Output data is again in a flat, contiguous array.
    // *NO* checks are made regarding size and fact count!
    uint32_t* data_out = eclair_get_facts(
      prog, reachable_fact_type
    );

    // Process results...
    for (uint32_t i = 0; i < fact_count; i++) {
      printf(
        "Reachable: (%d, %d)\n",
        data_out[i * 2], data_out[i * 2 + 1]
      );
    }

    // Cleanup of dynamic allocated data when no longer needed.
    eclair_free_buffer(data_out);

    // Cleanup of rest of the Eclair runtime.
    eclair_program_destroy(prog);
    return 0;
}

As you can probably tell by now, this API is much harder to use correctly than the other language bindings. It should only be used when no bindings exist for your language of choice, or when working in C or C++.

You can directly compile and link the above C code with the generated LLVM IR code as follows using clang:

$ clang -o program main.c path.ll

Other languages

Didn’t see your language in the list of supported language bindings? Open an issue or consider contributing your own language bindings. As long as your language supports a C FFI, you should be able to build your own bindings on top of them.

➜ Getting started with Eclair
Introduction to Datalog ➜
© 2021-2023 Luc Tielen