What is it used for?

Aspire as a content acquisition/processing/indexing platform offers different features for each different task, the main ones are listed here.

Content Acquisition with Connectors

• Built-in connectors to dozens of different data sources (see list of available connectors here)

  • Scalable:  Automatically distributes ingestion jobs across a cluster of nodes

  • Elastic:  Add and remove nodes at any time

  • Resilient:  Crawl state is carefully tracked at all points

    • Jobs on failed nodes are automatically picked up by other nodes

    • After a full system crash, crawling restarts from where it left off

  • High Performance:  Crawls are typically limited only by limitations on the source system

  • Incremental:  Automatically identifies incremental changes and processes only those changes

    • The method for detecting incremental changes is based on what is provided by the underlying content storage technology.

Content Publication with Publishers

• Built-in publishers to most commonly available search engines

  • Including but not limited to:

    • Elasticsearch

    • Solr

    • SharePoint

    • Google Cloud Search

    • Amazon Kendra

• Content migration publishers for Cloud-based storage solutions

• Real-Time Streaming systems such as

  • Amazon Kinesis

  • Apache Kafka

Metadata/Content extraction & manipulation

• Built-in components for many common content processing tasks

  • Such as text extraction, OCR, field mapping, domain mapping, archive file extraction, etc.

• Scripting for easy manipulation of metadata

• Document rendering as images (for thumbnail previews)

Document Level Security

• Fully understands document-level security

  • Ingests ACLs for each content source

  • Provides cached, high-performance group-expansion* for each content source

    • *group-expansion is a process where the user-group memberships are flattened in such a way that given any user, a flat list of its groups are listed, even the parent groups of the ones directly assigned to them.

      • For example

        • user Ann has been assigned to the Developers group only

        • group Developers is part of another group called IT_Operations

        • After the group-expansion process, Ann is listed to be part of both Developers and IT_Operations

  • Multi-domain identity extraction and mapping

  • Identity publication

Customizable components

Aspire is designed to be able to host and use independent components (connectors, publishers and content-manipulation are components), if there are no built-in components for what you need, Aspire provides a set of SDK frameworks to develop new components.

So you can:

• Create custom connectors and publishers

• Create custom pipelines and workflow controls

• Create custom components

Ease of deployment

• Components and configurations are deployed through Maven

• Properties allow for anything to be parameterized (e.g. server destinations, credentials, file directory locations, etc.)

• Content source configurations can be exported from any cluster and imported on another

• Container images for ease of deployment in Container orchestration tools such as Kubernetes