Skip to content
This repository was archived by the owner on Oct 12, 2026. It is now read-only.
This repository was archived by the owner on Oct 12, 2026. It is now read-only.

Create an implementation of an SQL encrypted query on a clear database using TFHE-rs #94

Closed

Description

@zaccherinij

Overview

TFHE-rs is a pure Rust implementation of TFHE for Boolean and integer arithmetics over encrypted data.

With this bounty, we are asking users to implement an SQL encrypted query on a clear database.

Here's a simplified overview of how FHE can be used to implement an SQL encrypted query on a clear database:

1. Query Encryption
The first step involves the user encrypting their SQL query using their FHE secret key before sending it to the server hosting the database. The encryption ensures that the server cannot see the actual query, only its encrypted form.

2. Processing Encrypted Query on Clear Database
The server then processes this encrypted query on the clear database. FHE computations mixing clear and encrypted data will return encrypted data, it means the server can perform the necessary SQL operations (like SELECT, JOIN, etc.) using the clear data as well as the encrypted query. The result of this operation is an encrypted version of the query result.

3. Decrypting the Result
Finally, the encrypted result is sent back to the user, who can decrypt it using their private key to obtain the clear text result of the SQL query.

We ask that you, the hunters, implement a database that can manipulate signed and unsigned integers (8, 16, 32, 64 bits), booleans and short strings (32 ASCII characters) as possible data types for a column.

As we are not looking at a classical database use case, we don’t expect the database to be resilient to crashes, be highly available etc. it is merely a datastore addressed in a similar way to a database via an encrypted SQL query.

How to participate?

1️⃣ Register here.
2️⃣ When ready, submit your code here.
🗓️ Submission deadline: May 12th, 2024.

Description

Expected Work

The implementation should live in its own crate (and not in tfhe-rs/tfhe/example) and depend on either tfhe-rs 0.5.x or the main branch of tfhe-rs.

We ask that your crate passes the clippy checks when treating warnings as errors.

cargo clippy -- --no-deps -D warnings

We ask you to provide both an executable and an API.

Executable

For the clear database storage you can use existing databases or tools but we require that you manage to load a database from a directory with the following structure:

db_dir

  • table_1.csv
  • table_2.csv

example of a table_content.csv:

column_1_name:uint32,colum_2_name:bool,column_3_name:string
0,true,"first line"
123,false,"some other line"

The executable takes as input a database to load with the format discussed earlier and a file containing the SQL query in plaintext to avoid dealing with escaping quotes on the command line, we require the following format for the output:

$ cargo run --release -- --input-db /path/to/db_dir --query-file query.txt
Runtime: 42.24 s
Clear DB query result: (some result)
Encrypted DB query result: (some result)
Results match: YES

erroneous results will automatically disqualify a submission.

API

You are free to use the High Level API with CPU or GPU or the integer API with CPU or GPU.

We only expect you to implement ONE API, implementing more APIs won’t improve you submission’s classification, correctness is mandatory, incorrect implementations will be disqualified, performance will be the main differentiator for submissions.

For the encryption we ask that you use a Select query from the sqlparser crate as input. You may choose another type if you explain how it's a better choice. You must return your own EncryptedQuery representation that can be used on the device of your choice.

There are 3 possible APIs to implement

  • CPU Integer API (uses tfhe::integer::ServerKey, an EncryptedQuery, a Table struct representing the clear database),
  • GPU Integer API (uses tfhe::integer::CudaServerKey and a CudaEncryptedQuery, a Table struct representing the clear database)
  • High-Level API CPU/GPU (uses tfhe::ServerKey, EncryptedQuery built on HL API types, a Table struct representing the clear database)

The APIs are specified in detail below, choose the one(s) you wish to implement (only implement methods of the API you chose)

CPU Integer API

fn default_cpu_parameters() -> PBSParameters;

fn encrypt_query(query: sqlparser::ast::Select) -> EncryptedQuery;

/// Loads a directory with a structure described above in a format
/// that works for your implementation of the encryted query
fn load_tables(path) -> Tables

/// # Inputs:
/// - sks: The server key to use
/// - input: your EncryptedQuery
/// - tables: the plain data you run the query on
///
/// # Output
/// - EncryptedResult
fn run_fhe_query(
    sks: &tfhe::integer::ServerKey, 
    input: &EncryptedQuery,
    data: &Tables,
) -> EncrypedResult;

/// The output of this function should be a string using the CSV format
/// You should provide a way to compare this string with the output of
/// the clear DB system you use for comparison
fn decrypt_result(clientk_key: &ClientKey, result: &EncryptedResult) -> String

GPU Integer API

fn default_gpu_parameters() -> PBSParameters;

fn encrypt_query_gpu(query: sqlparser::ast::Select) -> CudaEncryptedQuery;

/// Loads a directory with a structure described above in a format
/// that works for your implementation of the encryted query
fn load_tables(path) -> Tables

/// # Inputs:
/// - sks: The server key to use
/// - input: your CudaEncryptedQuery
/// - tables: the plain data you run the query on
///
/// # Output
/// - EncryptedResult
fn run_fhe_query_gpu(
    sks: &tfhe::integer::ServerKey, 
    input: &CudaEncryptedQuery,
    data: &Tables,
) -> EncrypedResult;

/// The output of this function should be a string using the CSV format
/// You should provide a way to compare this string with the output of
/// the clear DB system you use for comparison
fn decrypt_result_gpu(clientk_key: &ClientKey, result: &CudaEncryptedResult) -> String

High-Level API

/// Returns parameters and the device (CudaGpu/Cpu) on which the function /// should be ran
fn default_parameters() -> (PBSParameters, tfhe::Device);

fn load_tables(path) -> Tables

fn encrypt_query(query: String) -> EncryptedQuery;

// The server key is provided to allow setting up threads if needed
fn run_fhe_query(
    sks: &tfhe::integer::ServerKey, 
    input: &EncryptedQuery,
    data: &Tables,
) -> EncrypedResult;

fn decrypt_result(clientk_key: &ClientKey, result: &EncrypedResult) -> String

SQL support

We ask to be able to run the following encrypted queries:

Requested:
SQL Select: https://www.w3schools.com/sql/sql_select.asp
SQL Select Distinct: https://www.w3schools.com/sql/sql_distinct.asp
SQL Where: https://www.w3schools.com/sql/sql_where.asp
SQL And: https://www.w3schools.com/sql/sql_and.asp
SQL Or: https://www.w3schools.com/sql/sql_or.asp
SQL Not: https://www.w3schools.com/sql/sql_not.asp
SQL In: https://www.w3schools.com/sql/sql_in.asp
SQL Between: https://www.w3schools.com/sql/sql_between.asp

For integer values we expect you to manage the <, <=, >, >=, = operators
For strings we expect you to manage the = operator only

Note that the != (not equal) operator is built via the NOT operator of SQL and an = operator.

Note that some operators may re-use other implementations, for example the IN operator is indicated to be equivalent to multiple OR operators.

Bonus API:

SQL Joins: https://www.w3schools.com/sql/sql_join.asp

The format of the encrypted query and result are free, the evaluation of this bounty will be on speed and correctness on the Requested APIs, bonus APIs are NOT required and would only be used as a last resort to differentiate two submissions.

Other Details

Benchmarking

Implementations will be benchmarked on the following AWS hardware:
CPU: hpc7a.96xlarge (192 vCPU, 768 GB Memory)
GPU: p3.2xlarge (note that TFHE-rs only supports single GPU execution for now) (1 Tesla V100, 16 GB GPU Memory)

Parameters

The allowed parameters are:

PARAM_MESSAGE_2_CARRY_2_KS_PBS,
PARAM_MULTI_BIT_MESSAGE_2_CARRY_2_GROUP_2_KS_PBS,
PARAM_MULTI_BIT_MESSAGE_2_CARRY_2_GROUP_3_KS_PBS

Your solution must be able to run with the multi bit and non multi bit parameter sets, here it should be straightforward as the message and carry moduli are the same.

Note that the multi bit parameters should be faster on GPU, namely the group 3 should be the fastest. It could be the case on CPU as well but it can saturate even the largest CPU machines with threads and performance suffers as a consequence.

Turtle shell queries

As Mario Kart has its turtle shells, you will be able, if you want to, to provide one example of a small DB (with the directory and csv structure discussed earlier) and a “turtle shell” query you feel is very hard/has corner cases alongside your submission, we will run the DB and the query on the submissions we gather, corner cases beware! The performance and correctness on these hunter provided use cases will be used for the evaluation of the bounty, in addition to cases we’ll choose ourselves, we ask that the query runs in under 5 minutes on a laptop/commodity hardware, though our benchmarking machine are larger we want to keep runtimes reasonable when possible.

Good luck!

Reward

🥇Best submission: up to €5,000

To be considered best submission, a solution must be efficient, effective and demonstrate a deep understanding of the core problem. Alongside the technical correctness, it should also be submitted with a clean code, clear explanations and a complete documentation.

🥈Second-best submission: up to €3,000

For a solution to be considered the second best submission, it should be both efficient and effective. The code should be neat and readable, while its documentation might not be as exhaustive as the best submission, it should cover the key aspects of the solution.

🥉Third-best submission: up to €2,000

The third best submission is one that presents a solution that effectively tackles the challenge at hand, even if it may have certain areas of improvement in terms of efficiency or depth of understanding. Documentation should be present, covering the essential components of the solution.

Reward amounts are decided based on code quality and speed performance on a m6i.metal AWS server.

Related links and references

Questions?

Do you have a specific question about this bounty? Simply comment this issue and our team will get back to you!

Activity

  1. BraunPhilipp commented on Feb 12, 2024

    @BraunPhilipp

    Can we assume that the encrypted query is a tuple of multiple encrypted components parsed from the original SQL query?

    I saw a similar approach in the following paper for an equality comparison. Doing so will likely mean loosing some privacy with regard to access patterns. However, we can likely also deduce an access pattern based on using a clear database.

    https://eprint.iacr.org/2015/1176.pdf

  2. IceTDrinker commented on Feb 13, 2024

    @IceTDrinker
    Member

    @BraunPhilipp the nature of the query should remain concealed, the format after encryption is up to you

  3. matthiasgeihs commented on Feb 13, 2024

    @matthiasgeihs

    fn decrypt_result(result: &EncryptedResult) -> String probably needs a decryption key as input?

    As the result is a String, I assume the output is expected to be a string representation of whatever data has been selected?

    For example, if we run SELECT column_1_name, colum_2_name FROM table_content WHERE column_3_name="first line" on the example table above (table_content.csv), the output would be "0, true". Correct?

  4. IceTDrinker commented on Feb 13, 2024

    @IceTDrinker
    Member

    fn decrypt_result(result: &EncryptedResult) -> String probably needs a decryption key as input?

    As the result is a String, I assume the output is expected to be a string representation of whatever data has been selected?

    For example, if we run SELECT column_1_name, colum_2_name FROM table_content WHERE column_3_name="first line" on the example table above (table_content.csv), the output would be "0, true". Correct?

    you are right, that's an oversight, decryption needs the secret key, the string is just whatever equivalent form of the data to easily compare with your cleartext query (my guess is it's easier to stringify the result of the clear query rather than putting the encrypted result formatted as a DB entry)

  5. matthiasgeihs commented on Feb 13, 2024

    @matthiasgeihs

    If there is multiple rows that match, should we support returning multiple rows? Or just return one of the matching rows?

  6. IceTDrinker commented on Feb 13, 2024

    @IceTDrinker
    Member

    If there is multiple rows that match, should we support returning multiple rows? Or just return one of the matching rows?

    has to match the expected SQL behavior, so you should run the clear query on the SQL system you use as reference and check they are the same, so if the clear query returns several rows, then return the several rows

  7. matthiasgeihs commented on Feb 13, 2024

    @matthiasgeihs

    I don't think this works. You would need to hold enough space in the return value to hold the complete database. Reason: You may as well select the whole database, but the server can't know because it doesn't see the query. So it always needs to reserve enough space to potentially fit the whole DB into the result. Or am I getting something wrong?

  8. IceTDrinker commented on Feb 13, 2024

    @IceTDrinker
    Member

    I don't think this works. You would need to hold enough space in the return value to hold the complete database. Reason: You may as well select the whole database, but the server can't know because it doesn't see the query. So it always needs to reserve enough space to potentially fit the whole DB into the result. Or am I getting something wrong?

    No I think that if you do it to be completely oblivious that's correct, that's what we are looking at for now, we don't expect to run queries on super large DBs, I mean we know some implementations are impractical though they work (like the ecdsa signature we did a while ago)

  9. 123 remaining items

  10. 0xalexbel commented on May 12, 2024

    @0xalexbel

    Couple of questions:

    1. what does "Results match: YES" in the requested command line implementation ? Maybe you were planning to send the clear expected result so we can perform the result matching ?
    2. Encrypted DB query result: (some result) it might be huge ??
    3. What does Clear DB query result: (some result) exactly mean ? the raw decrypted query ? or the csv string ? or a pretty output ?
    4. what about the CSV result output format ? should we ouput '\r\n' or only '\n' as newline.
    5. I don't understand the requested API : encrypt_query is probably missing a client key as argument ? or am i missing something ??
    6. The name fn fhe_query_gpu(...) is a bit weird. shouldn't it be run_fhe_query instead (as listed in the other APIs)Don't want to be eliminated just because my API entries are incorrect...😅
  11. zaccherinij commented on May 12, 2024

    @zaccherinij
    CollaboratorAuthor

    @0xalexbel I don't think you'll get a reply from our dev team today! But I encourage you to submit your code before the due date in any case, as three submissions will be rewarded, and no mistake is eliminatory per se :) Best of luck. JZ

  12. zaccherinij commented on May 12, 2024

    @zaccherinij
    CollaboratorAuthor

    @0xalexbel I don't think you'll get a reply from our dev team today! But I encourage you to submit your code before the due date in any case, as three submissions will be rewarded, and no mistake is eliminatory per se :) Best of luck. JZ

  13. 0xalexbel commented on May 12, 2024

    @0xalexbel

    I am ready to submit but the link does not work !
    https://zama.ai/bounty-program
    Where is the submit link ?

  14. zaccherinij commented on May 12, 2024

    @zaccherinij
    CollaboratorAuthor
  15. 0xalexbel commented on May 12, 2024

    @0xalexbel

    Is it possible to secure the submission and still continue to write the README.md file explaining the whole algorithm and polish the doc. Since the deadline in 15h or so ?

  16. zaccherinij commented on May 12, 2024

    @zaccherinij
    CollaboratorAuthor

    @0xalexbel Yes, no worries. Our team will review submissions based on their latest PR at the submission deadline time. Meaning you can submit now, and edit your submissions until the submission deadline which is: tomorrow 2pm for Paris—FR time.
    Cheers.

  17. 0xalexbel commented on May 13, 2024

    @0xalexbel

    It's probably too late to ask the question, but still...

    Couple of questions:

    1. what does "Results match: YES" in the requested command line implementation ? Maybe you were planning to send the clear expected result so we can perform the result matching ?
    2. Encrypted DB query result: (some result) it might be huge ??
    3. What does Clear DB query result: (some result) exactly mean ? the raw decrypted query ? or the csv string ? or a pretty output ?
    4. what about the CSV result output format ? should we ouput '\r\n' or only '\n' as newline.
    5. I don't understand the requested API : encrypt_query is probably missing a client key as argument ? or am i missing something ??
    6. The name fn fhe_query_gpu(...) is a bit weird. shouldn't it be run_fhe_query instead (as listed in the other APIs)Don't want to be eliminated just because my API entries are incorrect...😅

    Note: I tried to interprete the best I could, hope I was right... 😅

  18. IceTDrinker commented on May 13, 2024

    @IceTDrinker
    Member
    1. it is to have an easy way to know if the program succeeded
    2. it's the clear result after encrypted processing this can be csv formatted
    3. same thing to be able to compare at a glance
    4. \n
    5. We may have missed the client key, you need a client key for sure
    6. We won't penalize people for that as it seems we made a mistake in the original topic
  19. 0xalexbel commented on May 13, 2024

    @0xalexbel
    1. it is to have an easy way to know if the program succeeded

    The program exits with error code 1 if something went wrong, like syntax error, wrong db format etc.

    ?? The program does not know if the SQL select is correct since it performed the request blindly
    Unless you are expecting us to run both a clear request along with an encrypted request and compare the 2 ??
    Am i missing something ??

  20. IceTDrinker commented on May 13, 2024

    @IceTDrinker
    Member

    the program does one request in the clear and one encrypted compares the two, I think it was made clear in the original post ?

  21. 0xalexbel commented on May 13, 2024

    @0xalexbel

    the program does one request in the clear and one encrypted compares the two, I think it was made clear in the original post ?

    I read the original instructions very carefully several times, there no mention of the comparison request. But i can add it...

  22. zaccherinij commented on May 13, 2024

    @zaccherinij
    CollaboratorAuthor

    Thank you to everyone who submitted to the Zama Bounty Program Season 5. Our team will review all submissions and give some initial feedbacks in the coming days!
    Cheers.

  23. moved this from Bounties [Season 5] to Awarded Contributions in Zama Bounty and Grant Program Overviewon May 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

🎯 BountyThis bounty is currently open📁 TFHE-rslibrary targeted: TFHE-rs

Type

No type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions