See also: Interactive Demo & Examples

Introduction

Filter specification objects are defined recursively as a finite-depth, single-rooted tree of namespace descriptor objects.

A namespace descriptor is a JavaScript object that defines constraints for a specific data namespace in data that is to be validated and normalized by a Filter Specification Processor (FSP).

Example 1

In this simple example the filter specification is defined using a single namespace descriptor object:

// Filter specification object directing the FSP to reject all input except string values.
module.exports = { ____accept: 'jsString' };

This reads "accept a string and reject everything else".

Filter reserves the "quanderscore" prefix for namespace descriptor directives that customize the behavior of the FSP algorithm for a specific namespace.

In the code above ____accept is a namespace descriptor directive that instructs the FSP to perform a type check and accept only strings as valid. The full set of directives is detailed below.

Example 2

In this example we declare a filter specification using several nested namespace descriptor objects to define a simple object schema.

// Filter specification object directing the FSP to reject all input except objects
// that have required property 'x' that is set to either a string or numerical value,
// and optional property 'y' that if specified must be an object with required sub-property
// 'z' that must be assigned a Boolean value.


module.exports = {
    ____types: 'jsObject',
    x: {
        ____accept: [ 'jsString', 'jsNumber' ]
    },
    y: {
        ____types: [ 'jsUndefined', 'jsObject' ],
        z: { ____accept: 'jsBoolean' }
    }
};

There are several key points in this expanded example:

• There are four namespace descriptor objects in this expanded filter specification example.
• Within each namespace descriptor, quanderscore-prefixed properties are directives that control the FSP for that namespace.
• Within each namespace descriptor, properties that are not quanderscore-prefixed are namespace descriptors for sub-namespaces.

Quanderscores

The reserved "quanderscore" (i.e. ____) prefix used for namespace descriptor directives was chosen as it's seldom used in actual data and is thus unlikely to conflict with the names of actual data namespaces/properties. This allows the format of the data to be mirrored in the filter specification making it simpler for developers to author and later read filter specifications as documentation. Also, this simplifies introspection as both the data and the filter specification can be accessed using the same dot-delimited object paths in JavaScript source. And, because the prefix is visually distinct, it's immediately obvious that you're reading a filter specification in source code. See also: Big Filter Spec Example.

Namespace Descriptors

Filter specification objects are single-rooted, finite-depth trees of namespace descriptor objects that are referenced by the Filter library's Filter Specification Processor (FSP) function to determine policy decision regarding validation and normalization of input data on a namespace-by-namespace basis.

Every namespace descriptor in a filter specification object must minimally define a type constraint.

The root namespace descriptor is always used by the FSP to determine validation/normalization policy for the root namespace of the input data passed to the FSP.

Namespace descriptor objects define a small collection of reserved, quanderscore-prefixed property names that are called directives. Directives customize the behavior of the FSP for a specific namespace in the input data passed to the FSP.

The quanderscore itself is reserved; it's an error to specify a quanderscore property anywhere in a filter specification that is not one of of the supported directives supported by the Filter library.

Namespace Descriptor Directives

Here is the complete list of supported namespace descriptor directives:

Using this table and the concepts introduced in the previous sections, we can now explore the details of using namespace descriptor directives to fully custom the per-data-namespace behavior of Filter's FSP.

Type Constraints

Every namespace descriptor object in a filter specification object declare a type constraint by specifying either the ____accept, ____types, or ____opaque directive. These are mutually exclusive options.

____accept is a string or array of string values in the set [ 'jsUndefined', 'jsNull', 'jsString', 'jsBoolean', 'jsNumber', 'jsObject', 'jsArray', 'jsFunction' ].

____types is a string or array of string values in the set [ 'jsUndefined', 'jsNull', 'jsString', 'jsBoolean', 'jsNumber', 'jsObject', 'jsArray', 'jsFunction' ].

____opaque is a Boolean value that if used is always set to true to indicate that a value of any type should be accepted by the FSP algorithm.

If an array of values is specified as the value of ____accept or ____types the type constraint is met if the type of the data value is one of the indicated set. If a single value is specified, then only a data value of that type is accepted.

____accept

____accept is used to verify the type of a container (object or array) and accept its content without further validation. For example, to accept an array of any value types we would write a filter specification that looks like this:

var filterSpec = { ____accept: "jsArray" };

Or, to accept any object...

var filterSpec = { ____accept: "jsObject" };

You may also use the ____accept type constraint directive on non-container atomic types such as strings numbers, Booleans... Note that when used for atomic types, ____accept and ____types (introduced below) are semantically equivalent and can be used interchangeably.

For example these two filter specifications are semantically equivalent:

var filterSpec1 = { ____accept: "jsString" };
var filterSpec2 = { ____types: "jsString" };

You are only allowed to use the ____accept for leaf namespace descriptors in your filter specification; It's an error to use this directive on a non-leaf node. For example, the following filter specification is invalid:

var badFilterSpec = {
    ____accept: "jsObject", // <- NO GOOD!
    x: {
        ____accept: "jsNumber"
    }
};

____types

Like ____accept the ____types type constraint directive specifies the the value type(s) allowed for a specific data namespace. However, ____types is more restrictive when applied to containers than ____accept insofar as the contents of the container are validated/normalized against the filter specification.

For example, to accept an array of numbers we would write a filter specification that looks like this:

var filterSpec = {
    ____types: "jsArray",
    element: { ____accept: "jsNumber" }
};

Or, to accept an object used as a structure (i.e. the names of the properties are known) we would write a filter specification that looks like this:

filterSpec = {
    ____types: "jsObject",
    description: {
        ____accept: "jsString"
    },
    dataPoints: {
        ____accept: "jsNumber",
    }
};

Type Constraint Nuances

If ____opaque is specified the constraint is considered to be met for any input and the FSP will splice a reference from the request into its result without further scrutiny unless ____defaultValue is also specified as a directive and the input happens to be undefined in which case the default value is used.

If ____accept or ____types is specified and does not include 'jsUndefined' in its array then it is required unless ____defaultValue is also specified as a directive and the input happens to be undefined in which case the default value is used.

If ____accept or ____types array does include 'jsUndefined' then its an error to also specify ____defaultValue because what the developer is saying is "accept nothing (undefined) as valid and leave it undefined in the result".

If ____accept or ____types is specified as the single value jsUndefined then the FSP will reject input data if it specifies any value for the namespace. For example:

var filterSpec = {
    ____types: "jsObject",
    bannedProperty: {
        ____accept: "jsUndefined" // any value specified for 'bannedProperty' will result in runtime error
    },
    allowedProperty: {
        ____accept: "jsString"
    }
};

Collections

JavaScript support two native types that are used for collections: the object and the array.

Arrays require no explanation. But, JavaScript objects are ambiguous because they can be used to model maps (associative arrays or dictionaries if you prefer) that use keys that we do not know in advance. And, JavaScript objects can also be used as as structures with key names that are predefined.

Objects Used as Structures

By default, Filter assumes that JavaScript objects are used as structures with known key names.

A simple filter specification example that accepts an object used as a structure looks like this:

// Filter specification: accepts an object with predefined key:value structure.
module.exports = {
    ____types: 'jsObject',
    name: { ____accept: 'jsString' },
    address: { ____accept: 'jsString' }
};

Objects Used as Maps/Associative Arrays/Dictionaries

To validate/normalize a JavaScript object used as a map (associative array or dictionary), we must explicitly configure the FSP to accept an arbitrary number of key(s) using the ____asMap directive:

// Filter specification: accepts an map/dictionary of strings.
module.exports = {
    ____types: 'jsObject',
    ____asMap: true,
    key: { ____accept: 'jsString' },
};

Note that the Filter library places restrictions on heterogeneous collections: arrays and dictionaries require that the element type be declared (unless declared using ____accept that forgoes validation/normalization). Filter does not currently support the declaration of fully-validated/normalized collections of different types of sub-collections.

Arrays

Arrays are handled in a manner that's very similar to objects used as a map/dictionary.

// Filter specification: accepts an array of numbers.
module.exports = {
    ____types: 'jsArray',
    element: { ____accept: 'jsNumber' }
};

Name Constraints

The FSP algorithm uses the name of each namespace descriptor object in a filter specification to determine which properties of a data namespace to examine. Namespaces that are neither explicitly declared nor implicitly accepted by a parent namespace in the filter specification are pruned by the FSP.

For example:

const arccore = require('arccore');


// Create a filter object via the filter factory.
var filter = arccore.filter.create({
    operationID: 'demo',
    inputFilterSpec: {
        ____types: 'jsObject',
        itemName: { ____accept: 'jsString' },
        itemCount: { ____accept: 'jsNumber' },
        itemData: { ____accept: [ 'jsObject', 'jsUndefined' ] }
    }
}).result;


var responses = {
    badInput: filter.request({}),
    mininumInput: filter.request({
        itemName: "apple",
        itemCount: 6 }),
    validInput: filter.request({
        itemName: "orange",
        itemCount: 12,
        itemData: { type: "citrus" }
    }),
    superfluousInput: filter.request({
        itemName: "cherry",
        itemCount: 64,
        superfluous: [ 1, 2, 3, 4, 5 ,6 ,7 ]
    })
};


console.log(JSON.stringify(responses, undefined, 4));

... produces the following output:

{
    "badInput": {
        "error": "Filter [td0sKTF2Tc2PM-p7oX8Plw::unnamed] failed while normalizing request input. Error at path '~.itemName': Value of type 'jsUndefined' not in allowed type set [jsString].",
        "result": null
    },
    "mininumInput": {
        "error": null,
        "result": {
            "itemName": "apple",
            "itemCount": 6
        }
    },
    "validInput": {
        "error": null,
        "result": {
            "itemName": "orange",
            "itemCount": 12,
            "itemData": {
                "type": "citrus"
            }
        }
    },
    "superfluousInput": {
        "error": null,
        "result": {
            "itemName": "cherry",
            "itemCount": 64
        }
    }
}

Note that in the last example, superfluousInput, the array of numbers passed into the filter via request.superfluous has been pruned from the response.result value returned by the filter because it is neither declared nor implicitly accepted by the input filter specification.

This behavior is designed to check the natural tendency of JavaScript applications to become a snarled mess of quickly-implemented, undocumented, and generally untested subsystem API's by forcing data consumers and producers to negotiate on requirements and maintain shared filter specifications for critical subsystem API contracts.

Default Values

Any namespace descriptor object in a filter specification may specify a constant value to be used in the specific case that the input for that namespace is undefined.

This is done by adding a ____defaultValue directive to the namespace descriptor object. For example:

var filterSpec = { ____accept: 'jsNumber', ____defaultValue: 5 };

There's quite a bit more to what can be done with default values than one might expect looking at the last trivial example.

Consider this more complex scenario that uses default values to affect some rudimentary preprocessing to get the request served up to bodyFunction on a silver platter.

var filterSpec = {
    ____types: 'jsObject',
    ____defaultValue: { x: 0, y: 5000 },
    x: {
        ____types: 'jsNumber',
        ____defaultValue: 5000
    },
    y: {
        ____types: 'jsNumber',
        ____defaultValue: 10000
    }
};


// undefined -> { x: 0, y: 5000 }
// {} -> { x: 5000, y: 10000 }
// { x: 7 } -> { x: 7, y: 10000 }
// { x: 7, y: 7, z: 99 } -> { x: 7, y: 7 }

There are several interesting aspects of this example worth noting:

• Default values may specify data that is applied to sub-namespaces. For example, if no input is provided to the above example, then the root namespace descriptor's default value is used.
• Once a default value is taken by the FSP for namespace, it is treated as if it was part of the original input data. This means that it is validated/normalized (and potentially defaulted) in sub-namespaces just as the input data would have been if it was actually specified.

Note that there are some nuanced restrictions on specifying a default value via the ____defaultValue directive that related to the specific type constraint declared for a data namespace. Read the Type Constraints section carefully if you cannot get the filter factory to accept your filter specification(s).

Value Constraints

After the input has passed through FSP's type constraint checks and possibly been assigned a default value, developers can further constrain what's considered to be valid by using either the ____inValueSet or ____inRangeInclusive directives.

All you need to know really is that:

• ____inValueSet is an array
• ____inRangeInclusive is an object with begin and end properties.

FSP uses standard comparison operators and doesn't get fancy when it comes to checking value constraints. This means you should really only rely on these constraints to work reliably (for others) when used on string and numerical types.

Here's a simple example that leverages both ____inValueSet and ____inRangeInclusive:

const require('arccore');
var factoryResponse = arccore.filter.create({
    operationID: 'demo',
    inputFilterSpec: {
        ____types: "jsObject",
        activity: {
            ____accept: 'jsString'
            ____inValueSet: [ 'running', 'walking', 'sitting', 'sleeping' ],
        }
        duration: {
            ____accept: 'jsNumber',
            ____inRangeInclusive: { begin: 0, end: 100 }
        }
    }
});