---json
{
    "page_id": "epjz4h4b1hldpe2ls2ylp"
}
---
====== Data Resource - Generator Manifest ======


===== About =====
A ''data resource generator'' is a [[docs:resource:content|content data resource]] that generates ''data''.






===== Syntax =====
A ''generator''  is a [[docs:resource:manifest|resource manifest file]]
  * terminated by the suffix ''--generator.yml''
  * where the ''metadata'' field contains the [[docs:resource:data-definition|data definition file]] syntax and adds:
      * the [[#max-record-count|max-record-count]] [[attribute|attribute]] (maximum number of record generated)
      * the [[#stream-record-count|stream-record-count]] [[attribute|attribute]]: number of record generated by ''poll'' in a [[:docs:flow:stream_pipeline|stream]]
      * and the ''data-supplier'' column attribute node where the [[docs:generator:data-supplier|data supplier properties]] are set.

Example:
<file yaml example--generator.yml>
kind: generator
# Spec follows the data definition format
spec:
  # The maximum number of record generated
  max-record-count: 30
  # The number of record generated for each resource in a generate stream supplier
  stream-record-count: 10
  # The columns definition
  columns:
    - name: columnName
      comment: A column with a sequence integer generator and its properties
      data-supplier: # the data supplier
        type: sequence
        arguments:
          start: 3
          step: 2
          maxTick: 5
   - name: columnName2
    ........
</file>

<note info>
See the [[docs:generator:data-supplier|column supplier page]] to see all type of generations that you can choose for a column.
</note>
===== Attributes =====
==== max-record-count ====
[[docs:generator:max-record-count|max-record-count]] is an [[attribute|attribute]] that defines the maximum number of record generated


==== stream-record-count ====

''stream-record-count'' is an [[attribute|attribute]] that defines the number of record generated in [[:docs:op:generate|generate stream operation]].

==== Count ====
The [[docs:resource:count|count attribute]] is calculated and defines how many record a generator would generate.

You can see it with the [[docs:tabul:data:info|data info command]].

Example with a [[docs:generator:sequence|sequence generator]], you would get the maximum value. For an integer, this is ''2147483647''
<unit >
<code bash>
tabul data info --strict-selection count--generator.yml@howto
</code>
<console>
Information about the data resource (count@memgen)
attribute             value                                                                                              description
-------------------   ------------------------------------------------------------------------------------------------   -----------------------------------------------------
MAX_RECORD_COUNT      100                                                                                                The maximum of records generated
SIZE                  100                                                                                                The size
SIZE_NOT_CAPPED       2147483647                                                                                         The number of records without max
STREAM_RECORD_COUNT                                                                                                      The records generated in a stream
ABSOLUTE_PATH         count                                                                                              The absolute path on the data system
ACCESS_TIME                                                                                                              The access time (access time)
COMMENT                                                                                                                  A comment
CONNECTION            memgen                                                                                             The connection name
COUNT                 100                                                                                                The number of records
CREATION_TIME         2025-11-10 19:48:59.159433321                                                                      The creation time (birth time)
DATA_URI              count@memgen                                                                                       The data uri
KIND                  generator                                                                                          The kind of media
LOGICAL_NAME          count                                                                                              The logical name
MD5                   ef69caaaeea9c17120821a9eb6c7f1de                                                                   The Md5 hash
MEDIA_SUBTYPE         vnd.tabulify.generator+yaml                                                                        The media subType
MEDIA_TYPE            text/vnd.tabulify.generator+yaml                                                                   The media type
NAME                  count                                                                                              The name of the data resource
PARENT                gen                                                                                                The parent
PATH                  count                                                                                              The relative path to the default connection path
SHA384                0c9b6656498be26d413bf3563198f01be3236d017f75943f9406922d08ba4ec137ffde15d2e95dcb4d77f9d6cd6eec79   The Sha384 hash
SHA384_INTEGRITY      sha384-DJtmVkmL4m1BO/NWMZjwG+MjbQF/dZQ/lAaSLQi6TsE3/94V0uldy013+dbNbux5                            The sha384 value used in the html integrity attribute
TABULAR_TYPE          data                                                                                               The tabular type
UPDATE_TIME                                                                                                              The last update time (modify time)
</console>
</unit>

==== MediaType ====

=== Manifest file ===

The [[docs:resource:media-type|media type]] of a file is ''text/vnd.tabulify.generator+yaml''


=== Manifest fragment ===
The [[docs:resource:media-type|media type]] of a fragment in a [[:howto:pipeline:define|define step (ie inline generator)]] should be ''text/vnd.tabulify.generator+yaml-fragment''


Example:
<unit>
<file yaml pipeline/define_single_generator.yml scroll="toggle" height="300px">
kind: pipeline
spec:
  steps:
    - name: "Define"
      operation: "define"
      args:
        data-resource:
          # The below media-type has `fragment` in its extension
          # It's a special media-type that makes it possible to define a generator as a yaml in a `define` step
          media-type: text/vnd.tabulify.generator+yaml-fragment
          data-def:
            logical-name: "my-sequence"
            max-record-count: 5
            columns:
              - name: "id"
                type: integer
                data-supplier:
                  type: sequence
    - name: "Print"
      comment: "Print the sequence"
      operation: "print"

</file>
</unit>

===== Creation =====

''generators'' can be created:
  * manually by creating a yaml file
  * automatically from pipeline input with the [[:docs:op:fill|fill data operation]]