guideCKEditor Cloud Services On-Premises

The On-Premises version of CKEditor Cloud Services is an application designed to be installed and run on the customer’s in-house servers and computing infrastructure, including private cloud. It is a highly scalable solution, designed to handle an unlimited number of concurrent connections. It contains all the features of CKEditor Cloud Services available as SaaS.

The documentation below refers to a simplified version of CKEditor Cloud Services On-Premises which was designed to be easy to setup and maintain (while preserving all features). It also lowers the running costs by reducing the number of servers required to run the whole application to the minimum.

If your usage scenario involves running CKEditor Cloud Services On-Premises for more than 50,000 concurrent connections, please contact us to discuss the most optimal setup for your needs.

The CKEditor Cloud Services On-premise installation requires two servers: one for running the application (“Application Server”) and another one for hosting the databases (“Database Server”).

# Preparing the Application Server

# Hardware requirements

For every 250 concurrent users CKEditor Cloud Services On-Premises requires 1 additional core of CPU and 500MB of RAM.

  • 1 core and 500 MB RAM support up to 250 concurrent users.
  • 2 cores and 1 GB RAM support up to 500 concurrent users.
  • 4 cores and 2 GB RAM support up to 1,000 concurrent users.
  • 8 cores and 4 GB RAM support up to 2,000 concurrent users.
  • 16 cores and 8 GB RAM support up to 4,000 concurrent users.

These calculations allow you to run CKEditor Cloud Services On-Premises without having to set up load balancers.

If you would like to run CKEditor Cloud Services On-Premises on multiple servers and use a load balancer, contact us for more details.

Note:

  • Concurrent user — A user who is active in a collaborative editing session. Idle users connected to the document do not increase the server load much.
  • The above configurations are able to handle an even larger number of active users, but due to the server load, the response times will be significantly higher and as a result the user experience will be affected.
  • The CKEditor Cloud Services On-premise collaboration server performance was checked on Amazon AWS, using EC2 “t2” instances.

# Software prerequisites (for the application installed directly on the server)

In order to install CKEditor Cloud Services On-Premises on the application server, the following software must be installed on the server first:

  • node.js – version 10 or newer: https://nodejs.org/en/download/package-manager/
  • pnpm – version 2.25 or newer: npm i pnpm@2.25 -g
  • pm2 – version 3.2.2 or newer: npm i pm2 -g
  • libpng-dev: sudo apt install libpng-dev
  • python – version 2.7 is recommended, version 3.x is not supported
  • gcc – version 4.9.x or newer
  • make
  • autoconf – 2.69 or newer
  • libtool – 2.4.2 or newer
  • pkg-config – 0.26 or newer
  • nasm – 2.10.09 or newer

On Ubuntu you can install gcc and make with build-essential:

sudo apt install build-essential

# Problem with node-gyp

In case of problems with node-gyp, please check the installation guide: https://github.com/nodejs/node-gyp#installation.

# Preparing the Database Server

# Hardware requirements

The minimum database server requirements (for MySQL and Redis installation) to handle up to 8000 concurrent users is 2 cores and 4GB of RAM (a1.large equivalent on Amazon EC2).

If your installation has to support more than 8000 users, contact us to discuss the most optimal hardware setup for your needs.

# Software requirements

The database server should have two database engines installed: Redis and MySQL.

# Redis

  • Version 3.2.6 or newer.
  • Redis stores all temporary data related to collaboration and is used as cache. The storage requirements for Redis depend on the number of active documents.
  • The average size of one document is around 500KB of memory in Redis but this size depends on many factors — the document length, the number of users, the number of changes, etc. and it can grow to many megabytes.
  • Use Redis Cluster if the application needs to handle a large number of active users. Please contact us if this is the case.

NOTE:

  • If you use AWS ElasticCache as a Redis database, you have to change the parameter group for Redis. Add the notify-keyspace-events option with the Ex value. You cannot modify the default parameter group for Redis, so do the following instead:

    • Go to ElasticCache Dashboard on AWS.
    • Go to Parameter Groups.
    • Click Create Parameter Group.
    • Select a correct Family group and fill in other fields.
    • Find your new Parameter Group and on the list below click Edit Parameters.
    • Find a notify-keyspace-events and change it to Ex.

    Have a look at AWS documentation for more information.

# MySQL

  • Version 5.6 or 5.7.

  • Create the database and a user with the following permissions: ALTER, CREATE, DELETE, DROP, INDEX, INSERT, SELECT, TRIGGER, UPDATE, LOCK TABLES, REFERENCES. Example database creation script:

    CREATE DATABASE `cs-on-premises`
      DEFAULT CHARACTER SET utf8mb4
      DEFAULT COLLATE utf8mb4_unicode_ci;
    
  • Change the max_connections setting to allow more connections:

    set global max_connections = 1000;
    

    Every instance of CKEditor Cloud Services On-Premises uses a few connection pools. Therefore, it is necessary to increase the maximum number of connections to the database.

  • MySQL stores all persistent data like comments, users, environments information, etc. The storage requirements for MySQL depend on many factors.

  • If you want to use other SQL database like Microsoft SQL Server, PostgreSQL, etc., please contact us.

# Setting up CKEditor Cloud Services On-Premises application

There are two ways to install and run the application:

# Setting up the application directly on the server

# Installation

Unpack the installation package on the Application Server.

  • Set execution permissions on the setup.sh file.
  • Run sh setup.sh.

# Running the application (directly on the server)

  • Create the configuration. Copy app/config.json to a safe place and fill it with your configuration data. See the configuration example.
  • Configure the logs directory — see Logs.
  • Go to app/.
  • Run:
    sudo CS_CONFIG_PATH=[path/to/your/config] pm2 start
    
    By default the application will be run using all available CPUs but it can be changed by setting the CS_INSTANCES variable:
    sudo CS_INSTANCES=1 CS_CONFIG_PATH=[path/to/your/config] pm2 start
    
  • Check PM2 logs to verify if the application starts properly:
    sudo pm2 logs cs
    
  • Create the Environment with a SecretKey via Environments Management App. Use this data to create token endpoint and run tests.
  • Run tests to verify if the system works properly — see Testing the application.

# Testing the application (for the application installed on the server)

Verify if the application works properly after the first run or an application update:

  • Go to tests/.
  • Run:
    CS_CONFIG_PATH=[path/to/your/config] CS_ENV_ID=[environmentId] CS_SECRET_KEY=[secretKey] npm run test
    
  • If there are no errors after finishing tests, the application is ready to use.

Note:

  • CS_CONFIG_PATH – The path to configuration data used in the Running the application section.
  • CS_ENV_ID and CS_SECRET_KEYEnvironment (with all Services added) and SecretKey created via Environments Management App. You can use existing data or create a new one dedicated for tests.
  • After tests you can delete the Environment dedicated for tests created by you and the Environments created by tests (with the e2e-test- prefix). This step is not required.
  • There is a possibility that some test will not pass. In this case, please run tests again.

# Application management

We recommend using PM2 software for application management. Below you can find a list of some of the most important commands:

  • pm2 restart cs – Restart all processes of the application.
  • pm2 reload cs – Reload all processes of the application, 0-seconds downtime.
  • pm2 logs cs --lines 1000000 – Stream the logs file.
  • pm2 scale cs <number> – Scale the number of processes up or down.
  • pm2 delete cs all – Remove all processes.
  • pm2 list – List all processes.
  • pm2 -h – Help.

More information can be found in the PM2 documentation.

# Configuration example

{
    "mysql_host": "127.0.0.1",
    "mysql_user": "root",
    "mysql_password": "password",
    "mysql_database": "cs-on-premises",
    "redis_host": "localhost",
    "redis_port": 6379,
    "redis_db": 1,
    "redis_password": "password", // optional
    "easyimage_storage_root_location": "/var/cs/easyimage/"
    "application_http_port": 8000,
    "logging_level": "error",
    "environments_management_secret_key": "secret"
}

Note:

  • Please save you configuration outside the application package directory. This prevents from overwriting your configuration during the application update process.
  • Please set easyimage_storage_root_location outside the application package directory. This prevents from unwanted changes or deletions in your images made during the application update process.

# Logs

Make sure that write permissions are granted for the logs directory.
By default all logs are stored in the /var/log/cs/cs.log file but it is strongly recommended to change this location by using the CS_LOGS_PATH variable:

sudo CS_LOGS_PATH=[path/to/logs] CS_CONFIG_PATH=[path/to/your/config] pm2 start

Note:

  • Please set CS_LOGS_PATH outside the application package directory. This prevents from unwanted changes in your logs made during the application update process.

# Generating a startup script

# Upgrading to a new version

# Very safe
  • Unzip the package with a new version to a separate directory, for example: new_version_dir.
  • Compare your current configuration with new_version_dir/app/config.json. If the schema properties changed, apply these changes.
  • Change application_http_port to some free port, not used in the currently running application.
  • Go to new_version_dir/ and run the installation — see Installation.
  • Go to new_version_dir/app and start the test application:
    sudo CS_CONFIG_PATH=[path/to/your/config] pm2 start src/index.js --name test_cs
    
  • Run tests — see Testing the application.
  • If the new version of the application works properly, shut it down:
    sudo pm2 delete test_cs
    
  • Change application_http_port to the previous port used in the application.
  • Go to new_version_dir/app/.
  • Stop the current version of the application and run the new version:
    sudo pm2 delete all && sudo pm2 kill && sudo CS_CONFIG_PATH=[path/to/your/config] pm2 start
    
  • Verify if everything is working correctly.
  • If yes, you can delete the previous version. If not, stop the new version and run the previous version again.
# Less safe
  • Unzip the package with a new version to a separate directory, for example: new_version_dir.
  • Compare your current configuration with new_version_dir/app/config.json. If the schema properties changed, please apply these changes.
  • Remove all current version files and directories (your application will still work with PM2). Make sure that all sensitive data (logs, EasyImage images, configuration) is outside current_version_dir:
    sudo rm -rf current_version_dir/*
    
  • Copy new files to current_version_dir:
    cp -a new_version_dir/* current_version_dir
    
  • Restart your application:
    sudo pm2 delete all && sudo pm2 kill && sudo CS_CONFIG_PATH=[path/to/your/config] pm2 start
    
  • Run tests — see Testing the application.

# Setting up the application inside the Docker container

# Running the application (inside the Docker container)

# Containerize example using docker
  • Go to the installation package root directory where the Dockerfile exists and build the docker image:
    docker build -t ckeditor-cs .
    
  • Launch the container:
    docker run --init -p 8000:8000 -v /path/to/your/dir:/var/cs/easyimage -e MYSQL_HOST=[your_mysql_host] -e MYSQL_USER=[your_mysql_user] -e MYSQL_PASSWORD=[your_mysql_password] -e REDIS_HOST=[your_redis_host] ckeditor-cs
    

Note:

  • You should always mount the docker volume -v /path/to/your/dir:/var/cs/easyimage to preserve uploaded files in case your Docker container gets removed.
  • Use environment variables to configure the application. See Docker container environment variables. Examples:
    • If an SQL database and Redis are hosted on 192.168.118.27, you should pass the address through MYSQL_HOST and REDIS_HOST variables:
      docker run --init -p 8000:8000 -v ~/easyimage_files:/var/cs/easyimage -e MYSQL_HOST=192.168.118.27 -e REDIS_HOST=192.168.118.27 -e MYSQL_USER=root -e MYSQL_PASSWORD=root_password ckeditor-cs
      
    • If you want to change the port on which the application will be served to 5000, override the APPLICATION_HTTP_PORT variable and change exposed ports:
      docker run --init -p 5000:5000 -v ~/easyimage_files:/var/cs/easyimage -e APPLICATION_HTTP_PORT=5000 -e MYSQL_HOST=192.168.118.27 -e REDIS_HOST=192.168.118.27 -e MYSQL_USER=root -e MYSQL_PASSWORD=root_password ckeditor-cs
      
  • You should override the ENVIRONMENTS_MANAGEMENT_SECRET_KEY variable using a unique and hard to guess string due to security reasons.
# Containerize example with local environment using docker-compose
  • Go to the installation package root directory where the Dockerfile exists.
  • Create the docker-compose.yml file:
    version: "2.2"
    services:
      mysql-database:
        environment:
          MYSQL_ROOT_PASSWORD: password
        image: mysql:5.6.38
        restart: always
        volumes:
          - ./init.sql:/docker-entrypoint-initdb.d/init.sql:ro
    
      redis:
        image: redis:3.2.6
        logging:
          driver: "json-file"
          options:
            max-size: "10m"
    
      ckeditor-cs:
        build:
          context: ./
        depends_on:
          - redis
          - mysql-database
        ports:
          - "8000:8000"
        restart: always
        init: true
        environment:
          MYSQL_HOST: mysql-database
          MYSQL_USER: root
          MYSQL_PASSWORD: password
          REDIS_HOST: redis
          ENVIRONMENTS_MANAGEMENT_SECRET_KEY: secret1
        volumes:
          - ~/easyimage_files:/var/cs/easyimage   
    
  • Create the init.sql file:
    CREATE DATABASE `cs-on-premises`
      DEFAULT CHARACTER SET utf8mb4
      DEFAULT COLLATE utf8mb4_unicode_ci;
    
  • Run docker-compose up.

Notes:

  • You should always mount the docker volume to preserve uploaded files in case your Docker container gets removed.
  • Please use environment variables to configure the application. See Docker container environment variables.
  • You should override the ENVIRONMENTS_MANAGEMENT_SECRET_KEY variable using a unique and hard to guess string due to security reasons.

# Docker container environment variables

These are environment variables that are used to configure the application when the container is started.

MYSQL_HOST                          - required
MYSQL_USER                          - required
MYSQL_PASSWORD                      - required
MYSQL_DATABASE                      - optional (default: "cs-on-premises")
REDIS_HOST                          - required
REDIS_PORT                          - optional (default: 6379)
REDIS_DB                            - optional (default: 1)
REDIS_PASSWORD                      - optional (default: "")
APPLICATION_HTTP_PORT               - optional (default: 8000)
LOGGING_LEVEL                       - optional (default: "error")
ENVIRONMENTS_MANAGEMENT_SECRET_KEY  - optional (default: "secret"), but it is highly recommeneded you change it 

# Testing the application (for the application inside the Docker container)

Verify if the application works properly after the first run or an application update:

  • Go to tests/.
  • Run:
    pnpm install
    
  • Run:
    APPLICATION_HTTP_PORT=[your_app_http_port] ENVIRONMENTS_MANAGEMENT_SECRET_KEY=[your_env_management_secret_key] CS_ENV_ID=[environmentId] CS_SECRET_KEY=[secretKey] npm run test
    
  • If there are no errors after finishing tests, the application is ready to use.

Note:

  • APPLICATION_HTTP_PORT and ENVIRONMENTS_MANAGEMENT_SECRET_KEY– Environment variables used to containerize the application in the Running the application inside the docker container section.
  • CS_ENV_ID and CS_SECRET_KEYEnvironment (with all Services added) and SecretKey created via Environments Management App. You can use existing data or create a new one dedicated for tests.
  • After tests you can delete the Environment dedicated for tests created by you and the Environments created by tests (with the e2e-test- prefix). This step is not required.
  • There is a possibility that some test will not pass. In this case, please run tests again.

# Environments Management

Manage the Environments in CKEditor Cloud Services On-Premises using the Environments Management console application.

# Environments Management features

  • Create, modify and delete Environments.
  • Add a Service to the existing Environment or remove it.
  • Add a SecretKey to the existing Environment or remove it.

# Running Environments Management (for the application installed on the server)

  • Go to the environments-management/src directory.
  • Run:
    CS_CONFIG_PATH=[path/to/your/config] node app.js
    

Note:

# Running Environments Management (for the application inside the Docker container)

  • Go to the environments-management directory.
  • Run:
    pnpm install
    
  • Go to the environments-management/src directory.
  • Run:
    APPLICATION_HTTP_PORT=[your_app_http_port] ENVIRONMENTS_MANAGEMENT_SECRET_KEY=[your_env_management_secret_key] node app.js
    

Note:

# Environments Management Usage

  • Manage the Environments by choosing available options from the menu by using Arrow and Enter keys from your keyboard.

# Endpoints to communicate with CKEditor Cloud Services On-Premises

  • WebSockets endpoint:
    http://localhost[:application_http_port]/ws/`
    
  • EasyImage upload endpoint:
    http://localhost[:application_http_port]/easyimage/upload/
    
  • Token endpoint:
    You should implement your own token endpoint using the data generated by Environment Management. Refer to the implementation example.

# SSL communication

You can communicate with CKEditor Cloud Services On-Premises using secure connections. To achieve this, you must set up a load balancer e.g. NGINX or HAProxy with your SSL certificate.
Below are HAProxy and NGINX configuration examples:

  • Basic HAProxy configuration:
global
    daemon
    maxconn 256
    tune.ssl.default-dh-param 2048

defaults
    mode http
    timeout connect 5000ms
    timeout client 50000ms
    timeout server 50000ms

frontend http-in
    bind *:80
    bind *:443 ssl crt /etc/ssl/your_certificate.pem
    redirect scheme https if !{ ssl_fc }

    default_backend servers

backend servers
    server server1 127.0.0.1:8000 maxconn 32
  • Basic NGINX configuration:
events {
    worker_connections  1024;
}

http {
    server {
        server_name your.domain.name;

        listen 443;
        ssl on;
        ssl_certificate /etc/ssl/your_cert.crt;
        ssl_certificate_key /etc/ssl/your_cert_key.key;

        location / {
            proxy_pass http://127.0.0.1:8000;

            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection "Upgrade";
            proxy_http_version 1.1;
        }
    }
}