Class DataSource

The base class for all DataSources, especially custom DataSources. It must always be inherited. It provides a lot of core functionality to get configurations, ensure caching and more.

Important: in most cases you will inherit the CustomDataSourceAdvanced DataSource for custom data sources.

Inheritance

System.Object

ToSic.Lib.Services.ServiceBase

ToSic.Lib.Services.ServiceBase<DataSource.MyServices>

DataSource

App

AppRoot

AttributeFilter

AttributeRename

CacheAllStreams

CustomDataSourceAdvanced

EntityIdFilter

EntityTypeFilter

Error

ItemFilterDuplicates

LanguageModeler

OwnerFilter

Paging

PassThrough

PublishingFilter

Query

QueryRun

RelationshipFilter

SerializationConfiguration

Shuffle

StreamFallback

StreamMerge

StreamPick

Attributes

MetadataTargetTypes

QueryInfo

Scopes

SystemStack

Zones

TreeModeler

ValueFilter

ValueSort

AdamFiles

CmsBlock

Implements

IDataSource

IDataSourceSource

IAppIdentity

IZoneIdentity

IAppIdentityLight

ICacheKey

ICacheExpiring

ITimestamped

ICanPurgeListCache

IHasLog

IDataSourceTarget

IDataSourceShared

Namespace: ToSic.Eav.DataSources

Assembly: ToSic.Eav.DataSources.dll

Syntax

[PublicApi]
public abstract class DataSource : ServiceBase<DataSource.MyServices>, IDataSource, IDataSourceSource, IAppIdentity, IZoneIdentity, IAppIdentityLight, ICacheInfo, ICacheKey, ICacheExpiring, ITimestamped, ICanPurgeListCache, IHasLog, IDataSourceTarget, IDataTarget, IDataSourceShared, IAppIdentitySync

Remarks

Had a major, breaking update in v15. Consult the guide to upgrade your custom data sources.

Constructors

DataSource(DataSource.MyServices, String)

Default Constructor, protected. To inherit this, make sure your new class also gets the MyServices in it's constructor and passes it to here.

Declaration

protected DataSource(DataSource.MyServices services, string logName)

Parameters

Type	Name	Description
DataSource.MyServices	services	All the services a DataSource needs. Uses the MyServices convention TODO: DOCUMENT
System.String	logName	Your own log name, such as My.CsvDs

DataSource(MyServicesBase<DataSource.MyServices>, String)

Declaration

protected DataSource(MyServicesBase<DataSource.MyServices> extendedServices, string logName)

Parameters

Type	Name	Description
MyServicesBase<DataSource.MyServices>	extendedServices
System.String	logName

Fields

MyConfiguration

Correct prefix to use when retrieving a value from the current data sources configuration entity. Always use this variable, don't ever write the name as a string, as it could change in future.

Declaration

public static readonly string MyConfiguration

Field Value

Type	Description
System.String

Properties

AppId

The app id as used internally

Declaration

public virtual int AppId { get; protected set; }

Property Value

Type	Description
System.Int32	The App ID this thing belongs to

CacheFullKey

Combination of the current key and all keys of upstream cached items, to create a long unique key for this context.

Declaration

public virtual string CacheFullKey { get; }

Property Value

Type	Description
System.String	Full key containing own partial key and upstream keys.

CachePartialKey

Unique key-id for this specific situation - could be the same for all instances, or could vary by some parameter.

Declaration

public virtual string CachePartialKey { get; }

Property Value

Type	Description
System.String	A string which is specific to this cache-item.

CacheRelevantConfigurations

Some configuration of the data source is cache-relevant, others are not. This list contains the names of all configuration items which are cache relevant. It will be used when generating a unique ID for caching the data.

Declaration

[InternalApi_DoNotUse_MayChangeWithoutNotice]
public List<string> CacheRelevantConfigurations { get; }

Property Value

Type	Description
System.Collections.Generic.List<System.String>

CacheTimestamp

System time-stamp of when the data in this cached thing was initialized or updated. Depending on the implementation, this may go up-stream and return an up-stream value.

Declaration

public virtual long CacheTimestamp { get; }

Property Value

Type	Description
System.Int64	A timestamp as a long number

Configuration

The configuration system of this data source. Keeps track of all values which the data source will need, and manages the LookUp engine which provides these values.

Declaration

public IDataSourceConfiguration Configuration { get; }

Property Value

Type	Description
IDataSourceConfiguration

Error

Declaration

[PublicApi]
public DataSourceErrorHelper Error { get; }

Property Value

Type	Description
DataSourceErrorHelper

Guid

Internal ID usually from persisted configurations IF the configuration was build from an pre-stored query.

Declaration

public Guid Guid { get; }

Property Value

Type	Description
System.Guid	The guid of this data source which identifies the configuration IEntity of the data source.

In

List of all In connections

Declaration

[PublicApi]
public IDictionary<string, IDataStream> In { get; protected set; }

Property Value

Type	Description
System.Collections.Generic.IDictionary<System.String, IDataStream>

Item[String]

Gets the Out-Stream with specified Name.

Declaration

public IDataStream this[string outName] { get; }

Parameters

Type	Name	Description
System.String	outName

Property Value

Type	Description
IDataStream	an IDataStream of the desired name

Exceptions

Type	Condition
System.NullReferenceException	if the stream does not exist

List

The items in the data-source - to be exact, the ones in the Default stream.

Declaration

[PublicApi]
public IEnumerable<IEntity> List { get; }

Property Value

Type	Description
System.Collections.Generic.IEnumerable<IEntity>	A list of IEntity items in the Default stream.

Name

Name of this DataSource - not usually relevant.

Declaration

public string Name { get; }

Property Value

Type	Description
System.String	Name of this source.

Out

Gets the Dictionary of Out-Streams. This is the internal accessor, as usually you'll use this["name"] instead.
In rare cases you need the Out, for example to list the stream names in the data source.

Declaration

[PublicApi]
public virtual IDictionary<string, IDataStream> Out { get; protected set; }

Property Value

Type	Description
System.Collections.Generic.IDictionary<System.String, IDataStream>	A dictionary of named IDataStream objects

ZoneId

ID of the zone (EAV Tenant)

Declaration

public virtual int ZoneId { get; protected set; }

Property Value

Type	Description
System.Int32	The zone ID this thing belongs to

Methods

Attach(String, IDataSource, String)

Add a single named stream to the In

Declaration

[PublicApi]
public void Attach(string streamName, IDataSource dataSource, string sourceName = "Default")

Parameters

Type	Name	Description
System.String	streamName	In-name of the stream
IDataSource	dataSource	The data source - will use it's default out
System.String	sourceName	The stream name on the source, will default to "Default"

Attach(String, IDataStream)

Add a single named stream to the In

Declaration

[PublicApi]
public void Attach(string streamName, IDataStream dataStream)

Parameters

Type	Name	Description
System.String	streamName	In-name of the stream
IDataStream	dataStream	The data stream to attach

Attach(IDataSource)

Attach a DataSource to In - replaces all existing in-streams.

Declaration

[PublicApi]
public void Attach(IDataSource dataSource)

Parameters

Type	Name	Description
IDataSource	dataSource	DataSource to attach

CacheChanged(Int64)

Detect if the cache has newer data. It's called using the TimeStamp of the dependent object which may still have old data.

Declaration

public virtual bool CacheChanged(long dependentTimeStamp)

Parameters

Type	Name	Description
System.Int64	dependentTimeStamp	New time stamp of a dependent object, which could have an older timestamp.

Returns

Type	Description
System.Boolean	True if the timestamps differ, false if it's the same

Remarks

This is implemented in each object, because sometimes it compares its own timestamp, sometimes that of another underlying object.

ConfigMask(String, String, Boolean)

Add a value to the configuration list for later resolving tokens and using in Cache-Keys.

Declaration

[PublicApi]
protected void ConfigMask(string key, string token, bool cacheRelevant = true)

Parameters

Type	Name	Description
System.String	key	The internal key to reference this value in the Configuration[Key] dictionary.
System.String	token	The string containing Tokens which will be parsed to find the final value.
System.Boolean	cacheRelevant	If this key should be part of the cache-key. Default is true. Set to false for parameters which don't affect the result or are very confidential (like passwords)

GetStream(String, String, Boolean, Boolean)

Gets the Out-Stream with specified Name and allowing some error handling if not found.

Declaration

[PublicApi]
public IDataStream GetStream(string name = null, string noParamOrder = "Params must be named (https://r.2sxc.org/named-params)", bool nullIfNotFound = false, bool emptyIfNotFound = false)

Parameters

Type	Name	Description
System.String	name	The desired stream name. If empty, will default to the default stream.
System.String	noParamOrder	see Convention: Named Parameters
System.Boolean	nullIfNotFound	In case the stream name isn't found, will return null. Ideal for chaining with ??
System.Boolean	emptyIfNotFound	In case the stream name isn't found, will return an empty stream. Ideal for using LINQ directly.

Returns

Type	Description
IDataStream	an IDataStream of the desired name

Remarks

1. Added in 2sxc 12.05
2. for more in-depth checking if a stream exists, you can access the Out which is an IDictionary

Exceptions

Type	Condition
System.NullReferenceException	if the stream does not exist and nullIfNotFound is false

ProvideOut(Func<IEnumerable<IEntity>>, String)

Provide a function to get the data which this DataSource offers.

This is the more generic IEnumerable implementation. We recommend using the IImmutableList overload as it allows the system to optimize more.

Declaration

[PublicApi]
protected void ProvideOut(Func<IEnumerable<IEntity>> getList, string name = "Default")

Parameters

Type	Name	Description
System.Func<System.Collections.Generic.IEnumerable<IEntity>>	getList	The function which will get the list.
System.String	name	(optional) stream name, defaults to Default

ProvideOut(Func<IImmutableList<IEntity>>, String)

Provide a function to get the data which this DataSource offers.

This is the ImmutableList implementation, which is recommended.

Declaration

[PublicApi]
protected void ProvideOut(Func<IImmutableList<IEntity>> getList, string name = "Default")

Parameters

Type	Name	Description
System.Func<System.Collections.Immutable.IImmutableList<IEntity>>	getList	The function which will get the list.
System.String	name	(optional) stream name, defaults to Default

PurgeList(Boolean)

Remove the current data from the cache, optionally also purge everything upstream

Declaration

public virtual void PurgeList(bool cascade = false)

Parameters

Type	Name	Description
System.Boolean	cascade	true to purge all sources as well, default is false

TryGetIn(String)

Get a specific Stream from In. If it doesn't exist return false and place the error message in the list for returning to the caller.

Usage usually like this in your GetList() function:

private IImmutableList<IEntity> GetList()
{
  var source = TryGetIn();
  if (source is null) return Error.TryGetInFailed(this);
  var result = source.Where(s => ...).ToImmutableList();
  return result;
}

Or if you're using Call Logging do something like this:

private IImmutableList<IEntity> GetList() => Log.Func(l =>
{
  var source = TryGetIn();
  if (source is null) return (Error.TryGetInFailed(this), "error");
  var result = source.Where(s => ...).ToImmutableList();
  return (result, $"ok - found: {result.Count}");
});

Declaration

[PublicApi]
public IImmutableList<IEntity> TryGetIn(string name = "Default")

Parameters

Type	Name	Description
System.String	name	Stream name - optional

Returns

Type	Description
System.Collections.Immutable.IImmutableList<IEntity>	A list containing the data, or null if not found / something breaks.

Remarks

Introduced in 2sxc 11.13

Implements

IDataSource

IDataSourceSource

IAppIdentity

IZoneIdentity

IAppIdentityLight

ICacheKey

ICacheExpiring

ITimestamped

ICanPurgeListCache

IHasLog

IDataSourceTarget

IDataSourceShared