Service setup

Installation guide for CREDEBL Platform Services

Before installing the CREDEBL Platform services, it is essential to ensure that your environment meets certain prerequisites. Properly setting up these requirements will help facilitate a smooth installation process and optimal performance of the platform. This includes preparing your system with the necessary software dependencies, configuring the required network settings, and ensuring adequate hardware resources. Following these guidelines will help prevent potential issues and ensure that the CREDEBL Platform services are installed and operate as intended.

System requirements

Here is the list of software needed to get started:

• Node.js and npm (>= 18)

• Git (>= 2.34.1)

• Docker (>= 24.0.5)

• Docker Compose (>= 2.20.3)

The above can directly be installed from their respective docs or from common prerequisites

---

Prerequisites

Here is the list of prerequisites software we will need for CREDEBL:

• Clone Repository

• Environment Variables

• PostgreSQL (>= 14)

• NATS (>= 2.6.4)

• REDIS (>= 7.4)

• Keycloak (>= 25.0.6)

• Sendgrid

• AWS S3

• Schema file server (optional)

• Agent Setup

Clone Repository

Clone the platform repository from GitHub:

git clone https://github.com/credebl/platform.git
cd platform

Environment Variables

To help you quick start, a .env.demo is already present at the root of the platform repository. To getting started, rename .env.demo to .env Please find your-ip in the .env file, and replace all occurrences it with your machine's Ip address.

If you want to know more about the environment variables, please refer to .env.sample file which is given at root of the repository.

Apart from the already present variables, you need to add few variables generated from the below prerequisites like the PostgreSQL, Keycloak, Sendgrid, AWS S3, etc at the respective steps.

PostgreSQL

In case you already have PostgreSQL preinstalled on your machine, you can simply skip the database setup and add the appropriate .env variable after creating user postgres and database credebl if not already created.

PostgreSQL is a powerful, open-source object-relational database system known for its robustness and advanced features. This guide provides instructions for installing and configuring PostgreSQL both natively and using Docker. Installing PostgreSQL

Initially we'll need to install postures on the host or on docker

Ubuntu

sudo apt install postgresql

# To verify the status of the PostgreSQL service, use:

sudo systemctl status postgresql

Docker

# Pull the PostgreSQL Docker image:
docker pull postgres

# Create a Docker volume to persist PostgreSQL data:
docker volume create CREDEBL

Access the PostgreSQL and create user and database

Ubuntu

# Access the PostgreSQL command line interface:
sudo -u postgres psql

# Create a new user and set password:
CREATE USER postgres WITH ENCRYPTED PASSWORD 'postgres';

# Create db:
CREATE DATABASE credebl;

# Grant all privileges on the database to the user
GRANT ALL PRIVILEGES ON DATABASE credebl TO postgres;

Docker

# Run the PostgreSQL Docker container:
docker run --name CREDEBL -e POSTGRES_PASSWORD=postgres -d -p 5432:5432 -v CREDEBL:/var/lib/postgresql/data postgres

# Enter the container
docker exec -it CREDEBL psql -U postgres

# Create a db
CREATE DATABASE credebl;

# Grant all privileges on the database to the user
GRANT ALL PRIVILEGES ON DATABASE credebl TO postgres;

# List all databases to confirm
\l

# Exit the PostgreSQL prompt:
\q

Optional: In case you customize the database details, you need to update this

Add the appropriate postgres environment variable to the .env file

.env

WALLET_STORAGE_HOST=your-ip
WALLET_STORAGE_PORT=5432 # postgres port
WALLET_STORAGE_USER='postgres' # postgres user
WALLET_STORAGE_PASSWORD='postgres' # postgres password 

# The format for below is as follows: postgresql://{postgres.user}:{postgres.password}@{your-ip}:{postgres.port}/{database-name}
POOL_DATABASE_URL="postgresql://postgres:postgres@your-ip:5432/credebl"
DATABASE_URL="postgresql://postgres:postgres@your-ip:5432/credebl"

You can read more about prisma connection URLs in PostgreSQL here: https://www.prisma.io/docs/orm/overview/databases/postgresql

NATS

The CREDEBL platform utilizes NATS for message-driven data exchange between its micro-services.

For default setup, you can skip NATS and REDIS configurations and Continue here

NATS configurations

1. NATS Configuration File (nats-server.conf) The configuration file defines how the NATS server operates, including port assignments and WebSocket support.

nats-server.conf

port: 4222
max_payload: 4194304  # 4 MB in bytes
websocket {
  port: 443
  no_tls: true
}

• port: The main port for NATS communication is set to 4222.

• max_payload: Sets the maximum message payload size to 4 MB.

• websocket: Enables WebSocket support on port 443 without TLS (for unsecured communication).

1. docker-compose.yml File

• The docker-compose.yml file will define the NATS service, map the necessary ports, and bind the configuration file.

docker-compose.yml

version: '3'
services:
  nats:
    container_name: nats
    image: nats
    command: ["-c", "/nats-server.conf"]
    ports:
      - '4222:4222'
      - '6222:6222'
      - '8222:8222'
    volumes:
      - ./nats-server.conf:/nats-server.conf:ro

Currently the .env.demo already contains NATS keys and the local setup skips authorization for NATS messaging between services. Refer here to know more about NATS authorization

To create your Nkeys, you can refer NATS tool nk as per their official documentation

• container_name: Sets the name of the container to nats.

• image: Uses the official NATS Docker image.

• command: Passes the NATS configuration file to the container at startup.

• ports: Exposes three key ports:

  • 4222: Main NATS client communication port.

  • 6222: Routing port for NATS clusters.

  • 8222: HTTP monitoring port for server statistics and health checks.

• volumes: Mounts the nats-server.conf configuration file into the container as read-only (ro).

1. Start the NATS Server

To start the NATS server, run the following command from the directory containing the docker-compose.yml file:

docker compose -f docker-compose.nats.yml up -d

REDIS

The CREDEBL platform leverages Redis as an in-memory data store, primarily for caching and queuing.

You can skip REDIS setup if you are installing Platform services using docker. Continue here

REDIS configurations

To set up Redis for the CREDEBL platform, follow the steps below to create and launch a Redis instance using Docker.

1. Create the docker-compose.yml File The docker-compose.yml file is used to define and configure the Redis container for the platform.

docker-compose.redis.yml

version: '3'
services:
  redis:
    image: redis:6.2-alpine
    restart: always
    ports:
      - '6379:6379'
    command: redis-server --save 20 1 --loglevel warning
    volumes:
      - cache:/data

volumes:
  cache:

1. Start REDIS Server Once the docker-compose.redis.yml file is in place, start the Redis service by running the following command:

docker compose -f docker-compose.redis.yml up -d

Keycloak

Keycloak is an open source identity and access management solution

1. Run Keycloak using docker:

docker run -d -p 8080:8080 -e KEYCLOAK_ADMIN=admin -e KEYCLOAK_ADMIN_PASSWORD=admin quay.io/keycloak/keycloak:25.0.6 start-dev

This command installs and starts Keycloak at the specified endpoints, locally accessible at http://localhost:8080.

1. Once logged in:

   This section guides you through the initial setup and configuration of Keycloak through its administration console.

   1. Go to administration console

   2. Login using username: admin, password: admin.

   3. Click on the Keycloak dropdown on the top left and add a new realm as “credebl-platform” by clicking on 'Create realm'.

   4. Go to 'Clients' under 'Manage' in the sidebar on the left and create new client as a clientId = “adminClient”.

   5. Click on Next Button

   6. Set client authentication to ON mode.

   7. Set Service Accounts Roles to ON mode.

   8. Set Direct Access Grants to ON mode.

   9. Click on Next Button

   10. Add http://your-url/* (Eg.: http://localhost:3000/*) in Valid Redirect URIs as front-end url.
   11. Add http://your-url/ (Eg.: http://localhost:8080/) in Root URL.
   12. Add http://your-url/ (Eg.: http://localhost:8080/) in Web Origins.

   

   adminClient details

   1. Save the details

2. Service Account Roles Settings:

   This section outlines the configuration of service account roles to manage permissions and access control within Keycloak.

   

   assigned service role

   1. If not already inside, go to adminClient from the clients in the side bar.

   2. Go to the Service account role section.

   3. Click on the Assign role button.

   4. Select Filter by clients from the dropdown menu.

   5. Select the below roles from the Available Roles and click on the Assign button:

      • Create-client

      • Manage-client

      • Manage-users

      • Query-clients

      • Query-users

      • View-clients

      • View-users

3. Realm Roles Settings:

   This section covers the configuration of realm role settings within Keycloak.

Holder realm role

Under the credebl-platform realm, from the menu select realm-roles.

• Under the credebl-platform realm, from the menu select realm-roles.

• In realm-roles, click on Create Role and create a role with Role name=“holder”

When a new user registers on the platform we are assigning them a “holder” role from the realm.

1. SSO Session Settings: Set SSO Session Idle to 2 days. This is the expiration time of the refresh token if the user is idle on the platform. We can configure this as per our need. To navigate to SSO Session, go to Realm settings option on left sidebar under configure, now under the Sessions tab you can see SSO Session Idle.

2. Update the .env

   Update the .env file for the Keycloak details: To set the env variable KEYCLOAK_MANAGEMENT_CLIENT_SECRET, in credebl-platform realm, go to clients >> adminClient. Now in the Credentials tab, copy the client secret

Keycloak client credentials

.env

KEYCLOAK_DOMAIN=http://localhost:8080/
KEYCLOAK_ADMIN_URL=http://localhost:8080
KEYCLOAK_MASTER_REALM=master
KEYCLOAK_MANAGEMENT_CLIENT_ID=adminClient
KEYCLOAK_MANAGEMENT_CLIENT_SECRET=
KEYCLOAK_REALM=credebl-platform

Below is an optional step to add users and can be skipped

Optional: Add users manually

This section provides guidance on adding users manually to the Keycloak realm.

1. Go to users.

2. Add username: [e-mail] and email: [e-mail].
3. Enable user and email, then click on save.

4. Change the password in the credential section for user login.

5. Update the Keycloak userId in the users table in the local database.

6. Also update the client secret key in the .env file, which you will get in the credential section of adminClient in the client section.

7. Test the flow by logging in to platform admin.

SendGrid

Create a SendGrid account, generate an API key, and grant necessary permissions for sending emails.

Make a note of the API-key as well as the email used, as this will be laterused in our .env and during Installation

• Add the send grid key in the .env

.env

SENDGRID_API_KEY=your-API-key

AWS S3

To utilize all functionalities of CREDEBL, total of 3 S3 buckets are required for;

1. Storing Organization logo during creating and updating an organization

2. Bulk issuance of credentials

3. Storing connection URL generated from Agent and creating shortened URL

From the above mentioned, 1 and 2 can be skipped, if the respective functionality of adding organization logo and Bulk issuance is unused

.env

# 1. Used for Adding org-logo during org creation and update 
# Optional (Can be skipped if no image is added during org creation and updation)
AWS_PUBLIC_ACCESS_KEY=
AWS_PUBLIC_SECRET_KEY=
AWS_PUBLIC_REGION=
AWS_ORG_LOGO_BUCKET_NAME=

# 2. Used for Bulk issuance of credential
# Optional (Can be skipped if Bulk issuance is not used)
AWS_ACCESS_KEY=
AWS_SECRET_KEY=
AWS_REGION=
AWS_BUCKET=

# 3. Used for storing connection URL generated from Agent and creating shortened URL
# Required (As connecting to org requires Shortened url)
AWS_S3_STOREOBJECT_ACCESS_KEY=
AWS_S3_STOREOBJECT_SECRET_KEY=
AWS_S3_STOREOBJECT_REGION= 
AWS_S3_STOREOBJECT_BUCKET=

According to the AWS_S3_STOREOBJECT_BUCKET name, as per the AWS S3 path style, add domain to access objects from the bucket and save it, as it is utilized for the another .env variable

.env

# Please refere AWS to determine your bucket url
# https://docs.aws.amazon.com/AmazonS3/latest/userguide/VirtualHosting.html#path-style-access 
SHORTENED_URL_DOMAIN='https://s3.AWS_S3_STOREOBJECT_REGION.amazonaws.com/AWS_S3_STOREOBJECT_BUCKET'

Note: Usually, SHORTENED_URL_DOMAIN for bucket names with dot ( . ) in it comes after '/': https://s3.AWS_S3_STOREOBJECT_REGION.amazonaws.com/bucket-name. While others without a dot(.) are often referred as a subdomain: https://bucket-name.AWS_S3_STOREOBJECT_REGION.amazonaws.com

Schema File Server

Clone Repository

Clone the platform repository from GitHub:

git clone https://github.com/ayanworks/schema-file-server.git
cd schema-file-server

Environment Variables

To help you quick start, a .env.demo is already present at the root of the platform repository. To getting started, rename .env.demo to .env Please find JWT_TOKEN_SECRET in the .env file and replace its value with a base64-encoded JWT token.

Note: You can create JWT Token using the following command: node -e "console.log(require('crypto').randomBytes(32).toString('hex'))"

Service Setup

1. Build the Docker Image

   docker build -t schema-file-server .

2. Run the Docker Container

   docker run -d --env-file .env -p 4000:4000 -v "$PWD/app/schemas:/app/schemas" --name schema-file-server schema-file-server

   • -d: Run container in detached mode

   • --env-file .env: Load environment variables from the .env file

   • -p 4000:4000: Map host port 4000 to container port 4000

   • -v "$PWD/app/schemas:/app/schemas": Mount the local app/schemas directory into the container

   • --name schema-file-server: Assign a name to the container

   • schema-file-server: The name of the Docker image

Update .env and agent.env

• Update the schema file server token and URL in both the .env and agent.env files.

.env

SCHEMA_FILE_SERVER_URL='http://localhost:4000/schemas/'
SCHEMA_FILE_SERVER_TOKEN=

agent.env

SERVER_URL='http://localhost:4000'
FILE_SERVER_TOKEN=

Agent Setup

The Docker image built during this process is used to launch agents for new organizations on the CREDEBL platform.The Docker image name is set as an environment variable during the platform setup in a later step.

The default image for agent is ghcr.io/credebl/credo-controller:latest refering to latest release of the credo-controller. Refer here: https://github.com/credebl/credo-controller/pkgs/container/credo-controller

Optional: In case you want to build your own image for the agent

Note: Currently to have agent setup(image built), we need node version to be - 18.19.0 However, we are actively seeking contributions to update it to the Latest Stable Version, you can check the open issue here

• Clone the following repository:

git clone https://github.com/credebl/credo-controller
cd credo-controller

• Use following commands to install and upgrade Yarn:

npm install -g yarn

• To install the dependencies use following command:

yarn

• Build Agent controller using the following command:

yarn build

• Create Docker Image using following command :

docker build . -t credo-controller-0.5.3:latest

Mention credo controller version or any tag name. Note the same version will also be needed in the .env against the variable AFJ_VERSION

• Add the Docker Image tag in the .env

.env

AFJ_VERSION=credo-controller-0.5.3:latest

Installations

Make sure the .env file is set with all the required environment variables as per the sample file and the env guide give above.

Before you start the services make sure to update the credebl-master-table.json present at location, libs/prisma-service/prisma/data sendgrid details as well as your ip-address

credebl-master-table.json

{
  "platformConfigData": {
    "externalIp": "192.168.x.x",
    "inboundEndpoint": "192.168.x.x",
    "username": "credebl",
    "sgApiKey": "API-key-received ",
    "emailFrom": "Mail used in sendgrid",
    "apiEndpoint": "http://192.168.x.x:5000",
    "tailsFileServer": "http://192.168.x.x:5000"
  },

At the root of the platfrom repo:

cd libs/prisma-service
npx prisma generate
npx prisma migrate deploy

Now seed the db, before starting the services using the following:

npx prisma db seed

Since, you are in the '/prisma-service', move back to the root

cd ../..

Start the services:

Docker setup

docker compose -f docker-compose-dev.yml up -d

To start container from pre-built docker images

In case we want to start services using the latest images from CREDEBL you can start services using the following command:

docker compose up -d

pnpm setup

Install the pnpm package using the following command if it isn’t already installed on your machine:

npm install -g pnpm

Install dependencies

pnpm i

To start the API gateway

pnpm run start

To start all microservices using below command followed by microservice name, example: start user service.

pnpm run start user

Services are as follows

utility connection ledger organization agent-provisioning (Only required once when provisioning base-wallet) agent-service issuance verification (Below are optional services) webhook geolocation notification cloud-wallet

Access the Platform API by navigating to http://localhost:5000