Table of Contents

Guide DataSources - Big Picture

you are here (click to zoom) - discover the stack

This should give you the big picture as to how DataSources work and interconnect.

Data and Streams in DataSources

  1. Every data-item is an IEntity containing data about a person, product, file-information etc.

  2. A list of these items is a IEnumerable<IEntity> which contains zero, one or many items

  3. A Stream is an object which has list of items and a name

  4. A correctly built Stream will Provide this list, but only run the code if it's requested

  5. Each DataSource has one or more named Out streams (the default is Default, others can have any name you want)

  6. Very advanced DataSources can have a dynamic list of Out Streams - like the App, which has a stream for each ContentType in the App

  7. Each DataSource can have one or more named In streams coming from other DataSources (usually the first one is called Default)

Building Data / Entities

  1. DataSources use ProvideOut to determine what Streams are provided

  2. The parameter of ProvideOut is a function which returns a list of items so it's either ProvideOut(FunctionName) or ProvideOut(() => { ... })

  3. The code of each ProvideOut is only run if the Stream is requested

  4. Different Base Classes have different ProvideOut signatures accepting different kinds of data

    1. The most basic base class only accepts functions returning IEnumerable<IEntity>
    2. Easier base classes accept any of IEnumerable<IEntity>, IEnumerable<RawEntity>, IEnumerable<AnonymousType> or even a single item (not an IEnumerable)

  5. Internally the data will always be converted to an IEnumerable<IEntity> so that it can be used by other DataSources

  6. When using the easier base classes, the conversion is done automatically

    1. You can also configure how the conversion works using options: new DataFactoryOptions(...) (see DataFactoryOptions)
    2. In very advanced cases you can also convert it to IEntity yourself, but this is not documented

  7. Generated IEntities can also have Relationships - like folders to files and visa versa
    This is not yet documented, but if you need this, check out the AppFiles DataSource

Receiving Data from In for further processing

If your DataSource performs filtering or similar actions on existing data, then this data comes in on the In streams. In such scenarios, you would simply iterate over the In[streamname].List and provide the result in your out-stream again. You can find many examples in the EAV DataSources code.

TODO:

Connections / Links

  1. When DataSources are attached to each other, they are connected by one or more Links

  2. Only the downstream DataSource (the one with the In) knows about the connection, the upstream DataSource (the one with the Out) doesn't know about it

  3. Data from connections are only accessed when necessary

  4. The links are also important for caching, since often caching depends on Upstream refreshes

Links use an immutable Fluid API, but this is not yet documented TODO:

Configuration of a DataSource

  1. DataSource classes can have many properties like int AmountToGet, bool IncludeDeleted etc. which change their behavior

  2. There is a Configuration System which automates how such configurations are passed to the DataSource

  3. To connect the Properties with the Configuration System you need the ConfigurationAttribute and Configuration.GetThis() - see Configurable DataSource

  4. Each DataSource has an own ConfigurationProvider, which gives the DataSource information about the environment (like Portal or Tab information), App-Settings and more

  5. The reason why each one has an own ConfigurationProvider is because each DataSource can have own configurations

  6. Each DataSource can have custom Configuration provided by code or entered by the user in VisualQuery

  7. Configurations are always string based since they use Tokens. Keep this in mind when you need int or bool, because conversions can fail, which is why Configuration.GetThis(fallback) has a fallback parameter - see ConfigurationProvider

  8. Tokens always use LookUp Sources so you should read about them

Configuration in VisualQuery

  1. When you use a DataSource in VisualQuery, you can configure it by clicking the cogwheel

  2. This is only available if the DataSource has a VisualQuery Attribute mentioning the ConfigurationType or if it's a Dynamic DataSource which automatically can find the configuration ContentType

  3. If the user edited data, internally it's stored as an IEntity object, which is then passed to the DataSource as a configuration

  4. Usually the fields in the UI are all string based, even for numbers and bool, because the user may want to use Tokens

Caching

  1. DataSources also have cache-identity mechanism, to inform any up-stream cache what parameters actually caused this result, so that the data could be cached if needed

  2. Configurations are taken into account when caching, so that the cache for AmountToGet=5 is different from AmountToGet=10

History

This document mentions too many different aspects of the system, so no detailed history is given here.

Edit this page