Settings File

Settings File

The settings.json file holds environmental information (server addresses, passwords, system properties, repository settings, etc.) for your Aspire installation.

What goes in the Settings File? This is the information to include in the settings.xml Configuration File:

  • Node configuration

  • Security configuration

  • Maven repositories

  • App Bundle properties

  • Applications to launch on startup

 

Settings File Location


On startup, the Aspire application will automatically attempt to load the settings from the configured provider. Aspire by default uses Elasticsearch as a provider, and the settings are expected to be in the aspire-settings index.

The settings file can be uploaded by using the aspire.bat/aspire.sh file with the command "-upload_settings <absolute_settings_path>"  or "-us<absolute_settings_path>" or in the case of using containers, these will be uploaded automatically on the startup. 

 

Structure of settings.json


The settings.json file contains sections for automatically starting system configuration files, setting Aspire system properties, and setting Apache Felix system properties. The structure is as follows:

{ "settings": { "authentication": { "tokenExpiration": "30m", "refreshExpiration": "1h", "type": "Ldap", "ldap": { "server": "ldap://oldap:389", "authentication": "simple", "bindDN": "cn=admin,dc=accenture,dc=com", "searchBase": "dc=accenture,dc=com", "userDNQuery": "(uid={user})", "groupsHoldMembers": "true", "memberAttr": "uniqueMember", "connectTimeout": "3000", "roles": [ { "dn": "cn=administrators,ou=Groups,dc=accenture,dc=com", "group": "true", "roles": [ "ADMINISTRATOR" ] }, { "dn": "cn=operators,ou=Groups,dc=accenture,dc=com", "group": "true", "roles": [ "OPERATOR" ] } ] } }, "configAdmin": { "properties": { "@pid": "org.apache.felix.webconsole.internal.servlet.OsgiManager", "property": [ { "@name": "username", "$": "admin" }, { "@name": "password", "$": "admin" }, { "@name": "manager.root", "$": "/osgi" } ] } }, "repositories": { "defaultVersion": "5.0-SNAPSHOT", "allowAutoUpdate": "true", "repository": [ { "@type": "distribution", "directory": "bundles/aspire" }, { "@type": "maven", "remoteRepositories": { "remoteRepository": { "id": "stPublic", "url": "https://accenturesca.jfrog.io/artifactory/public/" } } } ] }, "encryptionProvider": { "implementation": "com.accenture.aspire:aspire-encryption-provider", "jarName": "aspire-encryption-provider-5.0-SNAPSHOT.jar", "jarPath": "/bundles/aspire/", "className": "com.accenture.aspire.encryption.providers.AspireEncryptionProvider" }, "properties": { "property": [ { "@name": "sampleProperty1", "$": "http://localhost:8983" }, { "@name": "sampleProperty2", "$": "false" }, { "@name": "sampleProperty3", "$": "data/crawler" }, { "@name": "sampleProperty4", "$": "data" } ] }, "nodesProperties": { "worker": { "maxMemQueueSize": "1000", "queueSizeThreshold": "0.75", "cleanUpWaitTime": "300000", "cleanUpThreshold": "3600000", "maxEnqueueRetries": "5", "debug": "false", "appCleanUpWaitTime": "60000", "appCleanUpThreshold": "3600000", "tags": "", "entryProcessorBaseSleep": "200", "entryProcessorMaxSleep": "10000", "entryProcessorMaxIterations": "5", "entryProcessorMultiplier": "2", "batchLoaderBaseSleep": "200", "batchLoaderMaxSleep": "10000", "batchLoaderMaxIterations": "5", "batchLoaderMultiplier": "2", "connectionTimeout": "60000", "socketTimeout": "60000", "maxRetries": "3", "proxyHost": "", "proxyPort": "0", "proxyUser": "", "proxyPassword": "", "pingFrequency": "15000", "nodeFailureTimeout": "30000" }, "manager": { "scanBatchCreatorBaseSleep": "200", "scanBatchCreatorMaxSleep": "10000", "scanBatchCreatorMaxIterations": "10", "scanBatchCreatorMultiplier": "2", "processBatchCreatorBaseSleep": "200", "processBatchCreatorMaxSleep": "10000", "processBatchCreatorMaxIterations": "10", "processBatchCreatorMultiplier": "2", "crawlProgressManagerBaseSleep": "500", "schedulerBaseSleep": "10000", "maxBatches": "1000", "maxBatchItems": "100", "connectionTimeout": "60000", "socketTimeout": "60000", "maxRetries": "3", "proxyHost": "", "proxyPort": "0", "proxyUser": "", "proxyPassword": "", "pingFrequency": "15000", "nodeFailureTimeout": "30000", "tags": "", "workerRoundRobin": "false", "workerRoundRobinTimeout": "600000" } }, "autoStart": { "application": [ { "@config": "com.accenture.aspire:app-cf-bootloader" }, { "@enable": false, "@config": "com.accenture.aspire:app-admin-ui" } ] } } }

Auto Start Section


The "autoStart" section will automatically load applications when Aspire is initialized. It contains a simple list of application files to load, for example:

"autoStart": { "application": [ { "@config": "com.accenture.aspire:app-cf-bootloader" }, { "@enable": false, "@config": "com.accenture.aspire:app-admin-ui" } ] }

Applications are loaded in the order specified. However, since Aspire has component-dependency checking built-in, the order of load is usually not that important.

Both Application XML/JSON Files and App Bundles

Each application can be launched either from an application XML file or an App Bundle.

  • For application XML files: The @config attribute should hold the file name of the Application XML/JSON file to load.

  • For App Bundles: The @config attribute should hold the Maven coordinates of the App Bundle to start.

Rename Auto-Started Applications

In general, the name of the application will be taken as the “default name” as specified at the top of the application.xml file. 

However, you can specify other names for the configuration file using the @name attribute, as shown below:

{ "@name": "RDBConnector2", "@config": "com.searchtechnologies.appbundles:cs-rdbms-connector:2.0", "properties": { "property": [ { "@name": "rdbmsHasDefaults", "$": "false" }, { "@name": "debug", "$": "true" } ] } }

This lets you install the same App Bundle multiple times, but with different top-level names.

Application Properties

Finally, as shown above, applications can have a nested “properties” section which holds properties that are defined just for that application. These properties can then be used with the ${propName} substitution pattern within the application.xml file.

Repositories Section


The “repositories” identifies where to find component code to load into Aspire. 

"repositories": { "defaultVersion": "5.0", "allowAutoUpdate": "true", "maxVersion": "5.0.2", "repository": [ { "@type": "distribution", "directory": "bundles/aspire" }, { "@type": "S3distribution", "bucketName":"s3-repo",      "directory": "bundles/",   "regionName":"us-east-1", "accessKey":"optional", "secretKey":"optional" }, { "@type": "maven", "offline": "false", "localRepository": "~search/.m2/repository", "remoteRepositories": { "remoteRepository": { "id": "stPublic", "url": "https://accenturesca.jfrog.io/artifactory/public/" } } } ] }

 

The following options are available for the repositories section:

Property

Type

Default

Description

Property

Type

Default

Description

defaultVersion

string

LATEST

(Strongly Recommended) Specifies the default version for all artifacts for which no version is specified. Note that this defaults to a version of "LATEST" - but unfortunately, this has some odd behavior between the local and remote repositories (it only checks the local repository if the version is available on the remote repository, and the remote repository has been “scanned”).

allowAutoUpdate

boolean

true

(Optional) Enables updating the artifacts to the latest minor version available. The latest version depends on the version configured in the maxVersion option. 

maxVersion

string

none

(Optional) The max version supported for the artifacts.


There are three types of repositories that can be configured in the “repository” section:

Distribution Repository

The Distribution Repository will load the component Jar files in a directory within your Aspire distribution, typically the "bundles/aspire" directory.

It is configured as follows:

{ "@type": "distribution", "directory": "bundles/aspire" }

On startup, Aspire will scan through the entire directory looking for bundles to load. If at any time you add new bundles (or update bundles) in this directory, then click on "Check for Updates" on the Aspire application home page. This will cause Aspire to re-scan the directory so that the new files are available. The “directory” tag identifies the directory where the bundles can be located.

S3 Distribution Repository

The S3 bucket Distribution Repository will load the component Jar files to your Aspire distribution, typically from S3 "bundles/" directory. S3 directory must end with "/".

It is configured as follows:

{ "@type": "S3distribution",   "bucketName":"s3-repo",  "directory": "bundles/",  "regionName":"us-east-1", "accessKey":"optional", "secretKey":"optional" }

Upon startup, Aspire will scan the entire S3 directory, similarly to the Distribution repository, to look for bundles to load. If you add new bundles (or update existing ones) to this directory at any time, then click on "Check for Updates" on the Aspire application home page. This action will prompt Aspire to rescan the directory, making the new files available. The "directory" tag specifies the location where the bundles can be found.

Maven Repository

The Maven Repository loads the component Jar files directly from Maven. The Maven Repository allows Aspire to share the same Jars as Eclipse and the Maven command-line program. Therefore, any newly 'install'ed or 'deploy'ed Jar file artifacts will be automatically available to Aspire.

It is configured as follows:

{ "@type": "maven", "offline": "false", "localRepository": "~search/.m2/repository", "remoteRepositories": { "remoteRepository": { "id": "stPublic", "url": "https://accenturesca.jfrog.io/artifactory/public/" } } }

The following options are available for the maven repository:

Property

Type

Default

Description

Property

Type

Default

Description

localRepository

string

(user home directory)/.m2/repository

(Optional) Specifies the location of the Maven local repository, where jars will reside locally once they are downloaded from the remote repository. This is also the location where Maven “install” will install new or updated artifacts.

defaultVersion

string

LATEST

(Strongly Recommended) Specifies the default version for all artifacts for which no version is specified. Note that this defaults to a version of “LATEST”. Unfortunately, this has some odd behavior between the local and remote repositories: it only checks the local repository if the version is available on the remote repository, and the remote repository has been “scanned”.

offline

boolean

false

(Optional) Specifies if the system is “offline” - in which case the Maven repository will only ever look to the local repository for artifacts, and never the remote repositories.

Use Specific Versions of Bundle

If required, you can force the Maven repository to give you a specific version of a bundle. This is if you don't specify it in the factoryName in application.xml files or in the config attribute in the autoStart section of the settings file. In the settings.json file, the bundleVersions section must be created within the Repositories section.

Normally in Aspire, if references to Maven artifacts do not give the version, then the defaultVersion (see above) is used. However, you may add a bundleVersions section to the settings file to give more precise control over the versions of bundles loaded. The parameters are shown below:

Property

Type

Default

Description

Property

Type

Default

Description

bundleVersions\bundle\@groupId

String

com.accenture.aspire

(Optional) The group ID for the bundle to version

bundleVersions\bundle\@artifactId

String

 

Required: The artifact ID for the bundle to version

bundleVersions\bundle\@version

String

 

Required: The version of the bundle to request from Maven

If a requested bundle is not configured in the bundleVersions section, then the defaultVersion (as configured above) of that bundle will be requested.

If the version specified is not located in Maven, an error will occur.

 

Example:

The following snippet will load all requested bundles at version 5.0, except the three specified, which will be loaded at the requested version

"repositories": { "defaultVersion" : "5.2.2", "allowAutoUpdate" : "false", "bundleVersions": { "bundle": [ { "@artifactId": "aspire-tools", "@groupId": "com.accenture.aspire", "@version": "5.0.0.2-SNAPSHOT" }, { "@artifactId": "aspire-dbserver-source", "@groupId": "com.accenture.aspire", "@version": "5.0.0.1-SNAPSHOT" }, { "@artifactId": "aspire-adobe-experience-source", "@groupId": "com.accenture.aspire", "@version": "5.0.0.1-SNAPSHOT" } ] } }


Proxy Settings

You can configure Maven remote repositories to use an HTTP proxy for outgoing communications. This is useful when your Aspire server has restricted access to the Internet, and you want to be able to fetch bundles as normal from the configured repository. To do this, add a <proxy> section to your remote repository and set the following properties:

Property

Type

Default

Description

Property

Type

Default

Description

host

string

null

(Optional) Proxy server hostname or IP address.

port

string

0

(Optional) Proxy server port number.

user

string

null

(Optional) User required for authentication against the proxy server.

password

string

null

(Optional) Password for authenticating against the proxy server. You can encrypt it by following the instructions /wiki/spaces/aspire40/pages/68780988.

Example:

"remoteRepository": { "id": "stPublic", "url": "https://accenturesca.jfrog.io/artifactory/public/", "proxy": { "host": "127.0.0.1", "port": 8888, "user": "PROXY-USER", "password": "PROXY-PASSWORD" } }

Properties


Properties are specified as name/value pairs. For example:

"properties": { "property": [ { "@name": "sampleProperty1", "$": "http://localhost:8983" }, { "@name": "sampleProperty2", "$": "false" }, { "@name": "sampleProperty3", "$": "data/crawler" }, { "@name": "sampleProperty4", "$": "data" } ] }

Once specified in the settings.xml file, these properties become available for use in Application XML files. Careful use of such properties will make your system configuration files portable to multiple Aspire installations without modification.

You can use these properties from the UI when configuring content sources, services, and workflow applications.

For example, you might use "http://localhost:8080" as your SOLR server on your personal laptop, but then use "http://customer.searchtechnologies.com:8983" for the production site. Using a property allows the same system configuration file to be tested on one machine and then installed on another machine without modification.

Properties & Environment Variables in Application XML Files

Properties declared in the settings.xml file can be used in application XML files with the ${propertyName} syntax. As an example:

<component name="feed2Solr" subType="default" factoryName="aspire-post-xml"> <postXsl>config/aspire2solr.xsl</solrXsl> <postUrl>${solrServer}/solr/update</postUrl> </component>

In the above example, the "solrServer" property was defined in the settings.xml file and then referenced with ${solrServer} in the application XML file.

This property value substitution occurs automatically on the component configurations by the Component Manager. It does not require any further intervention or programming for any individual component.

The ${XXX} syntax can also be used for substitution of environment variables and Java system properties (i.e., those defined on the command line with -Dxxx=yyy). Substitution prefers properties defined in the settings.json file. If the property is not found in the settings.json file, the system properties are checked and if still not found, the system environment is checked. Furthermore, in these versions, properties may be defined from other properties:

"properties": { "property": [ { "@name": "baseDir", "$": "/home/user/aspire" }, { "@name": "configDir", "$": "${baseDir}/cfg" } ] }

Note: Property references for properties that are not in the settings.xml file will be left as-is. This allows for other configurations that use the same syntax (specifically, the Groovy Scripting component) to continue to operate properly.

Property Escaping (for Groovy Scripts)

If you need to insert a property into a Groovy script, assigning it to a string that contains the \ character (such as ${aspire.home} would), will cause Groovy to raise an error as it sees invalid escaped characters. To avoid this, you can prefix the property name with escape: and any \ characters in the contents of the property will be replaced with \\.

For example:

"properties": { "property": [ { "@name": "file", "$": "c:\top-directory\directory\file.html" } ] }

and

<config> <file>${file}</file> <escapefile>${escape:file}</escapefile> <fileattr attr="${file}">somevalue</fileattr> <escapefileattr escapeattr="${escape:file}">somevalue</escapefileattr> </config>

expands to

<config> <file>c:\top-directory\directory\file.html</file> <escapefile>c:\\top-directory\\directory\\file.html</escapefile> <fileattr attr="c:\top-directory\directory\file.html">somevalue</fileattr> <escapefileattr escapeattr="c:\\top-directory\\directory\\file.html">somevalue</escapefileattr> </config>

Properties for Applications


You can specify properties that apply to a specific application (rather than the properties above, which apply to all components).

<autoStart> <application config="config/system.xml"> <properties> <property name="debug">true</property> <property name="managerExternalRDB">false</property> <property name="managerRDB">CSRDB</property> </properties> </application> <application config="com.searchtechnologies.appbundles:cs-manager:4.0"> <properties> <property name="debug">true</property> <property name="managerExternalRDB">false</property> <property name="managerRDB">CSRDB</property> <property name="managerExternalJDBCUrl"></property> <property name="managerExternalJDBCDriverJar"></property> <property name="managerExternalJDBCUser"></property> <property name="managerExternalJDBCPassword"></property> </properties> </application> </autoStart>

 

These properties are passed to all components (and only those components) that exist “under” the component manager. If the same property names are used at both the global level and the component manager level, the component manager definition will be used for components “under” that manager, whilst the global value would be used for other components.

Apache Felix Configuration


Some Apache Felix configuration parameters can also be placed in the settings.xml file, as follows:

"configAdmin": { "properties": { "@pid": "org.apache.felix.webconsole.internal.servlet.OsgiManager", "property": [ { "@name": "username", "$": "admin" }, { "@name": "password", "$": "admin" }, { "@name": "manager.root", "$": "/osgi" } ] } }

Although, before using this approach, check to see if these parameters can be stored in the Apache Felix system properties file (called "felix.properties" for most Aspire installations). That may be the better location for these properties.

Inside <configAdmin>, each <property> tag contains a "pid" attribute, which is the “persistent ID” of the configuration element. The nested properties are the OSGi Configuration properties.

See the following for more information about OSGi and Apache Felix configuration properties:

Security Configuration


This is the configuration to use the Login page. 

"authentication": { "tokenExpiration": "30m", "refreshExpiration": "1h", "validateRemoteHost" : "true", "type": "Ldap", "ldap": { "server": "ldap://oldap:389", "authentication": "simple", "bindDN": "cn=admin,dc=accenture,dc=com", "bindDNPassword": "password", "searchBase": "dc=accenture,dc=com", "userDNQuery": "(uid={user})", "groupsHoldMembers": "true", "memberAttr": "uniqueMember", "connectTimeout": "3000", "readTimeout": "5000", "roles": [ { "dn": "cn=administrators,ou=Groups,dc=accenture,dc=com", "group": "true", "roles": [ "ADMINISTRATOR" ] }, { "dn": "cn=operators,ou=Groups,dc=accenture,dc=com", "group": "true", "roles": [ "OPERATOR" ] } ] } }

Important Information

In order to avoid issues of invalid token between nodes when security is enabled, the environment variable named ASPIRE_JWT_SECRET needs to be set with a random string, which is 32 characters (combination of numbers, special characters, letters lower and upper case) in length in every aspire node (all with the same value). 

Action available based on Role

Option

Operator

Administrator

Option

Operator

Administrator

Seed

Details menu option (Id, Crawl Id, View Metrics)

Crawl menu option (Full, Incremental, Identity and Reprocess)

Export option

New Seed

Details menu option (Id, Crawl Id, View Metrics)

Crawl menu (Full, Incremental, Identity and Reprocess)

Edit option

Clone option

Export option

Delete option

Workflow

Details menu option (Id)

Export option

New workflow

Details menu option (Id)

Edit option

Clone option

Export option

Delete option

Credentials

Details menu option (Id)

Export option

New Credentials

Details menu option (Id)

Edit option

Clone option

Export option

Delete option

Connections

Details menu option (Id)

Export option

New Connections

Details menu option (Id)

Edit option

Clone option

Export option

Delete option

Connectors

Details menu option (Id)

Export option

New Connectors

Details menu option (Id)

Edit option

Clone option

Export option

Delete option

Policies

Details menu option (Id)

Export option

New Policies

Details menu option (Id)

Edit option

Clone option

Export option

Delete option

Schedules

Details menu option (Id)

Export option

New Schedules

Details menu option (Id)

Edit option

Assign seed

Execute option

Clone option

Disable

Export option

Delete option

Tools

Extension Manager

Resource Manager

Export

Groovy Playground

DXF Playground

Javadoc

Debug Console

Extension Manager

Resource Manager

Import

Export

Groovy Playground

DXF Playground

Javadoc


Aspire uses a token-based security to access the Rest API. These are the general configuration options.

Property

Type

Default

Description

Property

Type

Default

Description

tokenExpiration

String

30m

(Optional) The access token expiration time

refreshExpiration

String

4h

(Optional) The refresh token expiration time. This value is recommended to be greater than the tokenExpiration

validateRemoteHost

boolean

true

(Optional) Whether or not the session tokens can only be used for the same node that generated it, and if the client IP should be recorded on the session for validation

type

String

 

Required: Currently only "Ldap" or "OIDC" authentication is supported.

LDAP Configuration

LDAP is configured as defined below.

When configuring Aspire behind a LoadBalancer make sure the health checks point to GET /aspire/_api/login endpoint which will always return a valid JSON if Aspire loaded correctly. If you use GET /aspire for health checks it will return 401 as it will be restricted as part of the authentication settings.

"type": "Ldap", "ldap": { "server": "ldap://oldap:389", "authentication": "simple", "bindDN": "cn=admin,dc=accenture,dc=com", "bindDNPassword": "password", "searchBase": "dc=accenture,dc=com", "userDNQuery": "(uid={user})", "groupsHoldMembers": "true", "memberAttr": "uniqueMember", "connectTimeout": "3000", "readTimeout": "5000", "roles": [ { "dn": "cn=administrators,ou=Groups,dc=accenture,dc=com", "group": "true", "roles": [ "ADMINISTRATOR" ] }, { "dn": "cn=operators,ou=Groups,dc=accenture,dc=com", "group": "true", "roles": [ "OPERATOR" ] } ] }

These are the configuration properties for the LDAP authentication:

Property

Type

Default

Description

Property

Type

Default

Description

server

String

 

Required: The LDAP server to use

authentication

String

anonymous

(Optional) The authentication types, “simple” and “anonymous” are supported

bindDN

String

 

Required: User DN to authenticate with. Not required if the authentication is anonymous.

bindDNPassword

String

 

Required: The password of the User DN. This value is recommended to be passed using the property aspire.ldap.bind.dn.password as a JVM parameter or as an environment variable instead of using the settings file. Not required if the authentication is anonymous. 

searchBase

String

 

Required: The base used to search the users to log in

userDNQuery

String

 

Required: The query used to search for the user.

groupsHoldMembers

String

false

(Optional) If true, the groups in LDAP contain the members

memberAttr

String

memberOf

(Optional) If groupsHoldMembers is true, this is the group attribute that contains the members. If groupsHoldMembers is false, this is the user attribute that contains the groups.

connectTimeout

String

1m

(Optional) LDAP server timeout in ms or using the ms, s, m, h units notation

readTimeout

String

5m

(Optional) LDAP read timeout in ms or using the ms, s, m, h units notation

roles

Array

 

Required: List of groups or users associated with roles.

roles/dn

String