Data Sources (IDataSource)

you are here (click to zoom) - discover the stack

Root DataSources are IDataSource objects which deliver one or many DataStreams, which contain a bunch of content-items.

Two Core Types

1. Root DataSources

This kind of DataSource generates or retrieves data from somewhere external like SQL, CSV, REST or the EAV-cache. This is a Root IDataSource.

An example is the Csv DataSource. Note that even this DataSource can have use Data on the In-Stream. But in this case the In provides configuration data, not raw data to process.

2. Processing DataSources

Processing DataSources receive data from another DataSource, then process/filter this and provide the result for further use, in which case it's both an IDataSource as well as an IDataTarget

An example of this is the Shuffle DataSource. It receives items on the In["Default"], randomly reorganizes them and provides them on the Out["Default].

DataSource Out can be another DataSources In

Datasources can be chained so that processing steps happen in a sequence. Here's a very simple example:

1. The root DataSource would be the App.Data which has all the data in the App
2. It can then be connected to a EntityTypeFilter DataSource which will limit the data to all BlogPost items
3. We can then connect it to a Shuffle DataSource to keep 3 random posts

This can be done both in code as well as in a prepared VisualQuery.

Queries: Reusable DataSource Wirings

As described above, DataSources can be joined together and the configuration can be saved as a Query. Here's another example:

1. a Root Csv DataSource can read a CSV-file and provide the data as a stream on Entities on csvDs["Default"]...
2. ...and pipe the result it to a CacheAllStreams which caches the data for 60 minutes
   this would ensure that the slow reading process only happens every hour
3. ...then pipe it to a ValueFilter, which only shows the items where the Country matches the Url-parameters country
4. ...then pipe it to a ValueSort, ordering it by LastName and then FirstName
5. ...then pipe it to another CacheAllStreams so that this common filter/sort combination will be kept for 5 minutes

The result can be used in a Template or streamed as JSON to a JavaScript SPA using the Headless API.

Understanding Data-Flow between DataSource Objects

Each DataSource has a list of out-streams available on the .Out["StreamName"] property, but usually access directly just with the DataSourceName["StreamName"]. This is what also happens when you use the Data object and write foreach(var item in Data["Default"]).

Aside from consuming data in your your template, most data-sources will simply offer the Out-Stream to other DataSources for further processing. Technically it's mapped like this:

• Cache.Out["Default"] > ContentTypeFilter.In["Default"]
• ... then some magic happens inside the ContentTypeFilter
• ... then the ContentTypeFilter.Out["Default"] has the resulting items, which can again be used as an In on another DataSource, or simply used in your template

Most DataSources will only have one In-stream and one Out-stream, but this is very variable depending on your need.

How to use

👉 Guide for Using DataSources in your C# Code

Create your own Custom DataSource

👉 Custom DataSources

Configuration of Each DataSource

The configuration uses a sophisticated token system to provide all necessary information. It is explained here.

Read also

• about Data Streams
• Dynamic Code CreateSource(...)
• .net API
• Demo-App showing some coding of DataSources
• Blog about creating your own data-source

History

1. Introduced in 2sxc 04.00