User Guide
You are looking at the UrbanMind user guide. Hopefully it should provide all the information required for you to use UrbanMind on your own as much as possible without having to reach out for support.
Introduction
UrbanMind is, as you probably know by now, a European data space implementation. It strives to be simple to use and easy to get started with. It would also be appropriate to call it opinionated and quite minimalistic.
Even though it is generally quite generic and could be used in or adapted to various data space verticals UrbanMind is primarily a smart city data space. This means that some of its functionality was designed in such a way that it makes the most sense when used in urban environments (think visualizations, data source service selection, data models).
UrbanMind is a multi-tenant data space implementation. This means that a single UrbanMind instance can host multiple isolated tenants that share its computational resources.
One of the main selling points of UrbanMind is that participants do not necessarily have to install anything on their premises to begin participating in the data space. The primary UrbanMind instance essentially follows a Software as a Service (SaaS) service model.
Participants
UrbanMind's tenants are, following the Data Spaces Support Centre data space blueprint, called participants.
Onboarding
Participant onboarding currently starts with a manual action. Interested parties have to reach out to one of UrbanMind's governance authorities. Instructions can be found on the access request page, a link to which can be found at the bottom of the login form.
OpenID Connect
Once a participant has been drafted they will have to create an OpenID Connect (OIDC) client for UrbanMind in their identity and access management (IAM) solution. Users with access to that client will have access to UrbanMind.
During the OIDC client creation the following URL should be whitelisted for redirects. This is the URL the participant's IAM solution will redirect the user to after a successful login flow.
After the client is created its connection details will have to be entered into UrbanMind so that a relationship can be established between the IAM and UrbanMind. This step can be performed by an operator or by the new participant themselves through a special participant setup form a link to which they can receive on request from the operator that is handling the onboarding for them.
The information that will have to be entered is the following:
- client ID
- client secret
- discovery URL (
.well-known)
In case that you decide for the operator to complete this step for your participant make sure to use a secure communication channel when sharing the information, since it is sensitive and could be abused by a third party.
Roles
UrbanMind has a set of roles which get assigned to a participant during their onboarding process.
| Role | Description |
|---|---|
| Data Producer | Provides data to the data space. For example, a municipality sharing traffic or environmental data. |
| Data Consumer | Uses data from the data space to build applications, run analytics, or create visualizations. |
| Participant Administrator | Manages users and access rights for their organization. |
| Operator | Manages the overall data space. Responsible for technical configuration, onboarding, and policy enforcement. |
For each assigned role there is an Expr rule that can be adjusted by an operator or the participant's own administrator. By default all the users that have been granted access to UrbanMind through the OpenID Connect setup will get assigned all the roles that have been whitelisted for the participant to distribute amongst its users.
Profile
Users of onboarded participants have access to a profile page on which they can see which information UrbanMind stores about them, i.e. which email address it uses as the default for notifications, which roles they got assigned, etc.
Impersonation
Participants with the operator role have the ability to temporarily impersonate other participants in the UrbanMind instance. Impersonation works on a participant level and not on a user level.
The functionality is meant to be used during support procedures and when a new participant is being onboarded.
It is properly audited and should not be overused, but there are cases where it will be the most sensible solution to a problem.
Offboarding
TBA
Services
Unlike some other data space implementations, UrbanMind does not let its participants offer arbitrary services to other participants in the data space. Instead, all services that are listed in the service catalog are offered by UrbanMind itself.
The offered services primarily work as data source adapters. Let's say that a participant has a PostgreSQL database and would like to expose a subset of its data through UrbanMind - they would instantiate the PostgreSQL service which would produce a reusable service integration.
Note: Participants can request the development of additional service adapters by contacting the UrbanMind developers through the support form or by sending an email to the support email address.
Service Integrations
The majority of services have two configuration forms, one for the service itself and one for resource distributions. You can find out more about resources their distributions in the sections below.
The service configuration (integration) form is used to collect all the service specific information required by UrbanMind to be able to interact with the service. This includes various types of service URLs, ports, headers, authentication, and authorization information. Basically anything that will be common to all the resource distribution configurations that will use the same service integration.
The service related subset of the resource distribution form is there to collect the rest of the information, which is unique to specific resources of the service (i.e. endpoints, buckets, tables). You can expect to be required to enter things like SQL queries and query parameters.
Resources
A lot of flows within UrbanMind revolve around resources. From the perspective of DCAT UrbanMind's resources correspond to the dataset entity. Resources can therefore represent a lot of things.
Examples:
- everything related to a specific project can be packaged into a resource
- a resource can represent the data from one's account in some application (service)
- a resource can be a collection of documents
Resources are the main building block of products, which are described in their own section below.
Distributions
Each resource can have one or more distributions. Distributions in UrbanMind correspond to distributions in DCAT.
The idea behind them is that the data relevant to a resource can be held on different locations and in different formats. It can use different data models, have different access policies, and so on.
Distributions can also represent different sample sizes (subsets) of the same data, be it split by time, geographical area, or anything else that can be used to divide data into sensible chunks.
One could also decide to provide a specific subset (sample) of the data for free and another (bigger, more valuable) subset against a payment. These could also be two different distributions of the same resource.
Distributions can use service integrations (recommended). Because service integrations are tied to distributions instead of resources a resource can include data from multiple different data sources.
A distribution without a service integration can not be consumed by UrbanMind. From UrbanMind's perspective it is merely a listing in its catalogue. Such distributions can not be exported or used (imported) within UrbanMind's use case functionality. If a resource (parent entity) of such a distribution does not provide a landing page URL for the distribution an access URL must be provided by the participant that provisioned the resource in UrbanMind.
Smart Data Models
UrbanMind uses and promotes the use of Smart Data Models. They are integrated into its resource distribution layer and into the DuckDB storage layer that is behind use cases.
When creating a distribution participants are encouraged to pick a Smart Data Model. If a Smart Data Model was picked for the distribution together with a variant of JSON for the media type the data will be validated against its JSON schema after the ingestion into UrbanMind. Invalid data will be discarded.
All Smart Data Models also have a official SQL schema file which is used by UrbanMind's use case DuckDB instances. Schemas are applied as migrations to each DuckDB instance at the creation of a use case. Data from distributions that have assigned Smart Data Models will be inserted into the appropriate Smart Data Model tables. Each Smart Data Model has its own table in the use case's DuckDB database. JSON data from distributions without Smart Data Models will be inserted into a generic catch-all table which means that it will still be queryable but the SQL queries will likely end up being a bit more complex than those using Smart Data Models where the data can be neatly distributed into columns for each property of the model.
In case that the service that was selected as the data source for the distribution does not provide the data in valid Smart Data Model format the user can use the embedded jq preprocessing engine to mold the data into a compatible shape.
Products
Products are vessels for sharing resources among participants of the UrbanMind data space. From the perspective of DCAT they correspond to the dataset series since resources correspond to datasets. Products could be considered the main building block of UrbanMind's data catalogue.
They are primarily a container entity and don't have much value on their own. They hold the metadata that makes resources discoverable.
A participant can have zero or more products. In case that a participant uses UrbanMind exclusively as a digital twin and not as a data space they might not find the need for products at all, since they are only meant for sharing and exposing data.
Contracts
Contracts express and control what a consumer of a set of resources can do after they have gained access to them through a product in UrbanMind's catalogue. A contract can be attached to multiple different products.
UrbanMind's contracts are modelled after ODRL. Even though they are stored in a different format at rest, participants can import and export contracts in ODRL's JSON encoding.
Contracts are evaluated using the embedded OPA engine.
Transactions
When a participant with the consumer role would like to obtain access to another participant's resources they have to request access to a product that contains those resources. This will initiate a transaction.
The producer will be notified and will have the option to accept or decline the transaction. As long as the transaction is active the consumer will have access to the resources according to the contract. Transactions can be cancelled by the product's producer.
Use Cases
Use cases are projects to which you attach accessible resource distributions. You select distributions data of which will be pulled into the database of a use case (DuckDB). The data is pulled in when the distribution is attached and can be periodically refreshed (existing data updated and new data inserted).
Widgets
The main idea behind use cases is that you collect the data of interest in a common data store, which you can then query against - explore, analyze, and create visualizations.
For visualizations UrbanMind's use cases provide the concept of widgets. Widgets include various types of charts, tables, and a REPL for more dynamic data exploration.
Extensions
DuckDB has the concept of extensions, which extend its core functionality. There are official extensions that are maintained by the DuckDB engineers and there are community extensions.
We've hand-picked a few of the extensions that we found relevant for UrbanMind and made it possible for them to be installed per use case. Each use case has its own isolated extension directory. Check the bottom of the page when you are looking at an individual use case.
Some of the extensions are installed by default when a new DuckDB instance (use case) is created. Extensions can not yet be removed.
Dashboards
Each use case that you create automatically gets a dashboard displaying all its widgets in a grid layout. The layout can be customized and the customizations are persisted for everyone to see. Layout changes currently affect all the users of the dashboard. Widgets can be arbitrarily positioned and resized. They wrap into a single column on smaller screens.
Exports
Some of the data that flows through or is held by UrbanMind can be exported and downloaded. This happens through a concept we've named exports. You can find all your participant's exports on the export page, a link to which can be found in the inventory dropdown in the navigation bar.
Right now the following entities support the export functionality:
- distributions
- resources
- use cases
In case that your user has the required permissions you should be able to find an export button near the top of the page when you are looking at an individual instance of the mentioned types.
Exports are asynchronous. What this means is that you can initiate the export process and walk away (work on other things) and UrbanMind can notify you (email) once the export is complete.
The exported data will be packaged in a ZIP archive. You should be able to find a download button for the archive on the export page. A link to the downloadable archive will also be sent to you in the mentioned notification email if it was entered during the creation of the export.
Exports are valid for an hour, after which they are marked as expired and the archive is removed.
Connectors
Please note that the connector functionality of UrbanMind is still being heavily worked on and should be considered experimental.
The idea behind data space connectors is that a participant installs one or more services on their premises which allow them to participate in a data space. Those services make up a data space connector. Different data space implementations have different connector implementations, which are not necessarily compatible between each other.
One of the main selling points of a data space connector is data ownership
UrbanMind does not (yet) have its own connector implementation that participants could install. Instead, it tries to support various existing and widely used connectors in such a way that a participant can remotely control some of the operations that they would otherwise have to perform through the UrbanMind web user interface or API.
API
In addition to the hypermedia API that powers the UrbanMind's web user interface UrbanMind also has a JSON API, sometimes called a REST API, which exposes some of its functionality in a more machine (automation) friendly format, enabling machine-to-machine (M2M) communication. The idea behind its JSON API is that clients can retrieve data and metadata from UrbanMind without using its web UI. This lets anyone with access to UrbanMind's JSON API build their own solutions around it, such as federated catalogue solutions, ETL pipelines that use UrbanMind as a data source, etc.
NGSI-LD
UrbanMind has its own catalogue implementation that was modelled after DCAT-AP. In addition to the web UI the catalogue is also exposed through a read-only subset of the NGSI-LD API using standardised DCAT-AP entities in JSON and JSON-LD formats.
The NGSI-LD API implementation is embedded into UrbanMind itself and is part of its JSON API.
Please note that the NGSI-LD API is currently limited to UrbanMind's catalogue but there is ongoing work that will extend the API to use cases, exposing the Smart Data Model data through a NGSI-LD interface.
Documentation
Links to UrbanMind's documentation can be found under the documentation dropdown in the navigation bar. Look for the open book icon, that is where you'll find it.
In addition to the UrbanMind landing page link where you can find some high-level details about the project itself there you should be able to find
- a link to the OpenAPI documentation for UrbanMind's JSON API,
- a link to this user guide,
- a link to UrbanMind's rulebook,
- a link to its privacy statement (PDF),
- and a link to the support form.
Chatbot
UrbanMind's participants should also be able to find a blue circle with a robot icon inside in the bottom right corner. Clicking on that circle should open up a chat window for interfacing with UrbanMind's chatbot.
The chatbot can assist you with UrbanMind's documentation, help you write various DuckDB SQL queries when you're building use cases, and interpret the results of those SQL queries.
Support
In case that you can't find the information you're looking for on any of the mentioned documentation pages feel free to send us an email to support@dataspace.urban-mind.eu.
Alternatively, in case that you're an existing participant, you can use the official support form which guides you through the process. As mentioned above, a link to the support form can be found in the documentation dropdown.