connector-period

Docker

The docker container is based on the image zenika/alpine-maven:3-jdk8. Uses Java JDK 8 and Maven 3. Configuration is composed of docker-compose.yaml and Dockerfile.

Dockerfile

Fundamental configuration which is used as a base for all services. Contains several variables:

Accessible in compile time:

• MAVEN_PROFILE: defines maven profile

Accessible in runtime:

• APP_PARAMS: defines parameters for an application
• DEBUG: defines if debugging will be enabled

Docker-compose

Configuration of services. This file contains configuration of each service where are defined build and runtime variables.

Name of service is fcs2 which is used for build and start the service. Build arguments contain definition of maven profile. Arguments variables defines in build section are accessible only during compile time. When the images is created, the variables can not be changed. Environment variables are located in environment section and can be changed after the build process. These variables are used to set configuration for the built. Debugging can be enabled to set DEBUG variable to true and publish the port 50005. Otherwise the debugging is disable.

docker-compose.yaml

services:
  fcs1:
    container_name: fieldclimateSenslog1
    build:
      dockerfile: Dockerfile
      context: .
      args:
         MAVEN_PROFILE: FieldClimateSenslog1
    ports:
      - "5005:5005"
    restart: always
    environment:
        APP_PARAMS: -cf config/fieldclimateSenslog10.yaml
        DEBUG: "true"

Build & run

To build all or a specific one service.

$ docker compose build
$ docker compose build <service_name>  // e.g. demoLogger

To start all or a specific one service.

$ docker compose up
$ docker compose up <service>          // e.g. demoLogger

To export docker image

$ docker save <image_name> > name.tar
$ docker save connector-period-senslogtelemetry2theros > connector_telemetry2theros.tar

To run docker image

docker run --name <container_name> -d --ulimit nofile=122880:122880 -e APP_PARAMS="-cf config/<config_name>.yaml" <image_name>
docker run --name senslog_connector_senslog2analytics -d --ulimit nofile=122880:122880 -e APP_PARAMS="-cf config/senslog1Analytics.yaml" connector-period-senslog2analytics

Java & Maven

The application supports Java 8+ and uses Maven as a build automation tool. Configuration for Maven contains profiles which defines modules that will be compiled.

It is recommended to clean entire project before compilation.

$ mvn clean

To build all modules.

$ mvn package

To build only the modules which contains modules for the Azure LoraWan and the Senslog V1.

$ mvn package -P LoraWanSenslog1

All modules (JARs) are compiled to the bin folder. To be able to start the application, it is important to add all JARs to classpath and to define main class (java ).

$ java -cp "bin/*" cz.senslog.connector.app.Main <app_params>

Parameters

The application allows these parameters:

Usage: General Senslog Connector [options]
    Options:
      -cf, -config-file
        Configuration file in .yaml format.
      -h, -help
 

Creating a new module

Intellij Idea

Right click on 'connector-period' -> New -> Module...

Module Settings

Name convention for a new module

pattern: connector-<fetch/push>-<module_name>
example1: connector-fetch-demo
example2: connector-push-demo

Build System: Maven
Parent: General Senslog Connector

Project Settings

Edit pom.xml file in the new module. The example is made for the module with the name 'fetch-demo'. Change the name according to the new name of created module. Add following attributes.

<artifactId>connector-fetch-demo</artifactId> # keep the generated name, should be chosen according to the name convention (see above)
<name>fetch-demo</name> # change the name according to a new name
<packaging>jar</packaging>

<dependencies>
        <dependency>
            <groupId>cz.senslog</groupId>
            <artifactId>connector-fetch-api</artifactId> # change the artifact to fetch or push according to the type of the module
            <version>${project.parent.version}</version>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-assembly-plugin</artifactId>
            </plugin>
        </plugins>
    </build>

Create a folder META-INF/services in resources to make following folder's structure:

src/main/resources/META-INF/services

Create a file with the name:

cz.senslog.connector.fetch.api.ConnectorFetchProvider

Java Settings

Create a package with following name convention (example is for fetch-demo):

cz.senslog.connector.fetch.demo

Create a java class with the name (example is for fetch-demo):

ConnectorFetchDemoProvider

Implement the interface in the Provider class:

cz.senslog.connector.fetch.api.ConnectorFetchProvider

Register the class (add following line) in the file in 'META-INF/services'

cz.senslog.connector.fetch.demo.ConnectorFetchDemoProvider

Config Settings

Create a new file in the './config' directory with the following name convention:

pattern: fetcherPusher.yaml
example1: demoSenslog.yaml
example2: myFetcherSenslog.yaml

Create a following structure:

settings:
    - Demo: # name of the fetcher module, e.g., Demo
        name: "Demo fetcher"
        provider: "cz.senslog.connector.fetch.demo.ConnectorFetchDemoProvider" 
        # the same line as in the file META-INF/services/cz.senslog...ConnectorFetchProvider

        # other attributes based on the module configuration

    - SenslogV1: # name of the pusher module, e.g., SenslogV1
        name: "Senslog V1"
        provider: "cz.senslog.connector.push.rest.senslog.v1.SenslogV1ConnectorPushProvider"

        # other attributes based on the module configuration

connectors:
    - DemoSenslogV1:        # name of the connector, e.g., DemoSenslogV1
        fetcher: "Demo"     # name of the fetcher from 'settings'
        pusher: "SenslogV1" # name of the pusher from 'settings'
        period: 86_400      # 24h 
        startAt: "02:30:00" # hh:mm:ss  # non-mandatory attribute
        initDelay: 5        # non-mandatory attribute

Maven Settings

Open the parent 'pom.xml' file of the project.

Check if the new module is registered in the following section:

<modules>
    ...
    <module>connector-fetch-demo</module>
</modules>

Create a new connector profile (the example is for creating a connector for Demo and Senslog):

<profiles>
    ...
    <profile>
        <id>DemoSenslog1</id>                           # profile identifier
        <modules>
            <module>connector-fetch-demo</module>       # name of the fetch module
            <module>connector-push-senslog-v1</module>  # name of the push module
        </modules>
    </profile>
</profiles>

Add the new module to the profile with the ID 'all'

<profile>
    <id>all</id>
    ...
    <modules>
        ....
        <module>connector-fetch-demo</module>
    </modules>
</profile>

Docker Settings

Register new service in the 'docker-compose.yaml' file:

  demoConnector:                            # id of the service
    container_name: demoSenslog1            # name of the running container
    build:
      dockerfile: docker/Dockerfile
      context: .
      args:
         MAVEN_PROFILE: DemoSenslog1        # name of the maven profile to build the connector
    ports:                                  # non-mandatory port for debugging
      - "5005:5005"
    restart: always                       
    environment:
        APP_PARAMS: -cf config/demoSenslog.yaml # path to the config
        DEBUG: "false"                      # non-mandatory param for debugging true/false

Create a model

Create a new package in the module 'connector-model' with the following name convention:

cz.senslog.connector.model.demo

Create a new class in the package with the name:

DemoModel

... and inherit the class AbstractModel:

public class DemoModel extends AbstractModel {
    protected DemoModel(OffsetDateTime from, OffsetDateTime to) {
        super(from, to);
    }  
}

Create a new model converter in the package 'converter', e.g. for the Demo and Senslog,

public final class DemoModelSenslogV1ModelConverter implements Converter<DemoModel, SenslogV1Model> {
    @Override
    public SenslogV1Model convert(DemoModel inputModel) {
        return null;
    }
}

Register the converter in the class:

cz.senslog.connector.model.converter.ModelConverterProvider

... in the method config():

register(DemoModelSenslogV1ModelConverter.class);