Monday, September 19, 2022

Show HN: Distributed JMAP and IMAP Servers in Rust

Stalwart JMAP Server

Stalwart JMAP is an open-source JSON Meta Application Protocol server designed to be secure, fast, robust and scalable. JMAP is a modern protocol for synchronising data such as mail, calendars, or contacts that makes much more efficient use of network resources.

Key features:

JMAP full compliance:
IMAP4 full compliance:
Robust storage:
- RocksDB backend.
- Full-text search support available in 17 languages.
- Blob storage for raw e-mail messages.
Secure:
Scalable and fault-tolerant:
- Node autodiscovery and failure detection over gossip protocol.
- Replication and cluster consensus over the Raft protocol.
- Read-only replicas.
- No third-party replication or cluster coordination software required.
Local Mail Transfer Protocol (LMTP) message ingestion.

Get Started

Install Stalwart JMAP on your server by following the instructions for your platform:

You may also compile Stalwart JMAP from the source.

Support

If you are having problems running Stalwart JMAP, you found a bug or just have a question, do not hesitate to reach us on Github Discussions or Discord. Additionally you may become a sponsor to obtain priority support from Stalwart Labs Ltd.

Documentation

Table of Contents

Get Started
Configuration
Management
Migration
Clustering
Development

Roadmap

The following major features and enhancements are planned for Stalwart JMAP:

Quota support
Filtering support (Sieve filters as well as other mechanisms)
JMAP Contacts, Calendars and Tasks support (currently IETF drafts)
Performance enhancements
Jepsen testing

Testing

Base tests

The base tests perform basic testing on different functions across the Stalwart JMAP code base. To run the base test suite execute:

Database tests

The database test suite performs a range of tests on the embedded key-value store such as:

Concurrent insertions.
Database queries.
Concurrent blob insertions and blob expiration.
Log compactions.

To run the database test suite execute:

cargo test store_tests -- --ignored

Core tests

The core test suite performs authorization, authentication and JMAP protocol compliance tests such as:

Access Control Lists (ACL) enforcement.
Authorization and Rate Limiting.
Event Source (JMAP Core).
OAuth authentication.
Push Subscriptions (JMAP Core).
WebSockets (JMAP over WebSocket).

To run the core test suite execute:

cargo test jmap_core_tests -- --ignored

Mail tests

The mail test suite performs e-mail and JMAP Mail protocol compliance tests such as:

Email/* functionality and compliance.
Mailbox/* functionality and compliance.
Thread/* functionality and compliance.
EmailSubmission/* functionality and compliance.
SearchSnippet/get functionality and compliance.
VacationResponse/* functionality and compliance.
Message thread id creation.
LMTP message ingestion.

To run the mail test suite execute:

cargo test jmap_mail_tests -- --ignored

Stress tests

The stress test suite generates concurrent random insert, get, query, update and delete operations on different JMAP datatypes to ensure data integrity. To run the stress test suite execute:

cargo test jmap_stress_tests -- --ignored

Another way of stress testing Stalwart JMAP is by using the IMAP stress test tool.

Cluster tests

The cluster test suite starts a Stalwart JMAP cluster consisting of five nodes and performs the following tests:

Distributed insert, read, update and delete operations.
Distributed e-mail thread merges.
Read replicas.
LMTP ingestion over RPC.
Raft elections.
Raft log conflict resolution.

To run the cluster test suite execute:

cargo test cluster_tests -- --ignored

Cluster fuzz

The cluster fuzz test suite starts a Stalwart JMAP cluster consisting of five nodes and attempts to corrupt the cluster state and/or its data by randomly performimg the following operations:

Crash current leader.
Crash follower.
Start offline follower.
Start all offline nodes.
Insert record.
Update record.
Delete record.

Please note that this test runs on a loop and does not stop unless a leader fails to be elected or a data corruption problem is found. If something goes wrong while running the tests, all the actions that were executed up to that point will be printed to screen as a JSON structure. This JSON dump can be later used to reproduce and eventually debug the problem.

To run the cluster fuzz test suite execute:

cargo test cluster_fuzz -- --ignored

Cluster Jepsen tests

Support for Jepsen testing is planned and in the roadmap. Until then, the clustering module will remain in beta.

JMAP test suite

Compliance with the JMAP protocol may also be tested using Fastmail's JMAP-TestSuite:

Clone the JMAP TestSuite repository:

git clone https://github.com/stalwartlabs/jmap-test-suite.git
cd jmap-test-suite/

Install the recommended Perl dependencies:

Run one of the tests, for example:

JMAP_SERVER_ADAPTER_FILE=eg/stalwart.json perl -I<PATH_TO_PERL_LIB> -I lib t/basic.t

Fuzz

To fuzz Stalwart JMAP server with cargo-fuzz execute:

 $ cargo +nightly fuzz run jmap_server

License

Licensed under the terms of the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. See LICENSE for more details.

You can be released from the requirements of the AGPLv3 license by purchasing a commercial license. Please contact licensing@stalw.art for more details.

Copyright

from Hacker News https://ift.tt/KQvWeHi

SymmetricalDataSecurity