![]() |
Qore Mapper Module Reference
1.5
|
this class is a base class for mapping data; see Mapper Examples for usage examples More...
Public Member Functions | |
| commit () | |
| Commits data written to the output data provider if the output data provider supports transaction management. More... | |
| constructor (hash< auto > mapv, *hash< auto > opts) | |
| builds the object based on a hash providing field mappings, data constraints, and optionally custom mapping logic More... | |
| discardOutput () | |
| Discards any buffered data in the output data provider in a bulk operation. More... | |
| flushOutput () | |
| Flushes any remaining data to the output data provider in a bulk operation. More... | |
| int | getCount () |
| returns the internal record count More... | |
| string | getFieldName (string fname) |
| returns a descriptive name of the given field if possible, otherwise returns the field name itself | |
| *AbstractDataProvider | getInputProvider () |
| returns the input provider | |
| *hash< string, AbstractDataField > | getInputRecord () |
returns the value of the "input" option | |
| MapperOutputRecordIterator | getOutputIterator () |
| returns an output record iterator that produces mapped data from the input data provider More... | |
| *AbstractDataProvider | getOutputProvider () |
| returns the output provider | |
| *hash< string, AbstractDataField > | getOutputRecord () |
returns the value of the "output" option | |
| auto | getRuntime (string key) |
| get current runtime option value for a key More... | |
| hash< string, hash< MapperOptionInfo > > | getUserOptions () |
| returns mapper options useful for users | |
| logOutput (hash< auto > h) | |
| calls the output logging closure or call reference (if any) to log the output record | |
| list< hash< auto > > | mapAll (list< auto > recs) |
| maps all input records and returns the mapped data as a list of output records More... | |
| list< hash< auto > > | mapAll (hash< auto > recs) |
| maps all input records and returns the mapped data as a list of output records More... | |
| *hash< auto > | mapBulk (hash< auto > rec, *hash< auto > crec) |
| maps a set of records in hash of lists format; returns mapped data in a hash of lists format More... | |
| hash< auto > | mapData (hash< auto > rec) |
| processes the input record and returns a hash of the mapped values where the keys in the hash returned are the target field names; the order of the fields in the hash returned is the same order as the keys in the map hash. More... | |
| hash< auto > | optionKeys () |
| returns a list of valid constructor options for this class (can be overridden in subclasses) More... | |
| replaceRuntime (*hash< auto > runtime) | |
| replaces runtime options More... | |
| resetCount () | |
| resets the internal record count More... | |
| rollback () | |
| Rolls back data written to the output data provider. More... | |
| runAutonomous () | |
| Runs the input and output mappers with data providers on each end autonomously. More... | |
| setRuntime (string key, auto value) | |
| set the runtime option with "key" to value "value" More... | |
| setRuntime (hash< auto > runtime) | |
| adds runtime options to the current runtime option hash More... | |
| hash< string, bool > | validKeys () |
| returns a list of valid field keys for this class (can be overridden in subclasses) More... | |
Static Public Member Functions | |
| static Type | recordType ("hash<string, AbstractDataField>") |
| the input and output record type | |
Public Attributes | |
| const | ConstantConflictList = ("name", "struct", "code", "default") |
| field keys that conflict with "constant" and "index" | |
| const | OptionKeys |
| constructor option keys (can be extended by subclassing and reimplementing optionKeys()) | |
| const | OutputKeys |
| output option keys | |
| const | UserOptions |
| User options. | |
| const | ValidKeys |
| default known mapper hash field keys (can be extended by subclassing and reimplementing validKeys()) | |
Protected Member Functions | |
| checkInputField (string k, string name) | |
| raises an error if an invalid input field name is declared; only call this if "input" is defined | |
| checkInputProvider () | |
| raises an error if no input provider is present | |
| checkMap () | |
| verifies the input map in the constructor | |
| checkMapField (string k, reference< auto > fh) | |
| perform per-field pre-processing on the passed map in the constructor More... | |
| checkOutputBulkOperation () | |
| raises an error if no output provider is present More... | |
| checkOutputProvider () | |
| raises an error if no output provider is present More... | |
| checkTimezoneOption (string rn) | |
| verifies a timezone constructor option | |
| constructor () | |
| private constructor for subclasses | |
| convertToHash (int t, string k, reference< auto > fh) | |
| convert a field definition to a hash if possible | |
| error (string fmt) | |
throws a MAP-ERROR exception; prepends the map name to the description if known More... | |
| error2 (string ex, string fmt) | |
| throws the given exception; prepends the map name to the description if known | |
| fieldLengthError (string k, string val, int ix, int sze, int maxlen, hash< auto > rc) | |
| called when a field exceeds its maximum length when processing hashes of lists | |
| *int | getRecListSize (hash< auto > rec) |
| hash< auto > | mapDataIntern (hash< auto > rec) |
| processes the input record and returns a hash of the mapped values where the keys in the hash returned are the target field names; the order of the fields in the hash returned is the same order as the keys in the map hash. More... | |
| nothing | mapFieldIntern (reference< hash< auto >> h, string key, hash< auto > rec, bool do_list, int list_size) |
| maps a single field to the target More... | |
| auto | mapFieldType (string key, hash< auto > mapping, AbstractDataProviderType type, auto value, hash< auto > rec) |
| performs type handling | |
| auto | mapSubclass (hash< auto > m, auto v) |
| to be overridden as necessary in subclasses | |
| setup (hash< auto > mapv, *hash< auto > opts) | |
| sets up the mapper object before checking the mapper hash | |
| string | truncateField (string k, string val, int ix, int sze, int maxlen) |
| called to truncate fields when processing hashes of lists | |
Protected Attributes | |
| bool | allow_dot = False |
DEPRECATED: do not assume struct when field names have a "." in them; instead allow input field names to have a "." in them. More... | |
| bool | allow_output_dot = False |
DEPRECATED: do not assume structured/hash output when output field names have a "." in them; instead allow output field names to have a "." in them. More... | |
| hash< auto > | consth |
| map of constant fields | |
| int | count = 0 |
| count of records mapped | |
| const | DeprecatedGlobalTransformOptionMap |
| maps deprecated transform options to global tranform options | |
| hash< auto > | global_transform_opts |
| global transformation options; can be overridden on a per-field basis | |
| hash< auto > | identh |
| map of fields to be mapped 1:1 input -> output | |
| *list< auto > | identl |
| list of fields to be mapped 1:1 input -> output | |
| *code | info_log |
| an optional info logging callback; must accept a sprintf()-style format specifier and optional arguments | |
| *hash< string, AbstractDataField > | input |
| an optional description of possible input hash keys | |
| bool | input_do_request |
| indicates that the request/response API should be used for mapper input | |
| *code | input_log |
| an optional input data logging callback; must accept a hash giving the input data hash | |
| *AbstractDataProvider | input_provider |
| the optional input data provider | |
| *hash< auto > | input_provider_search |
| search conditions for the input provider | |
| auto | input_request |
| the argument for the request/response API for the input provider | |
| *hash< auto > | input_request_options |
| options for the request when using the request/response API for the input provider | |
| bool | input_request_search |
| a boolean flag that indicates that an input provider that supports both the record API and the request/response API should use the request search API to provide records | |
| *hash< auto > | input_search_options |
| search options for the input provider | |
| *hash< auto > | m_runtime |
| current runtime values More... | |
| hash | mapc |
| the hash providing output field names and mappings More... | |
| hash< auto > | mapd |
| the hash with a subset of the mappings used dynamically | |
| hash< auto > | mapo |
| the hash of output records for key order | |
| *string | name |
| the optional name for the object (for example a table name); will be prepended to field names in error messages | |
| *hash< string, AbstractDataField > | output |
| an optional description of the output data structure | |
| bool | output_do_request |
| indicates that the request/response API should be used for mapper output | |
| *code | output_log |
| an optional output data logging callback; must accept a hash giving the output data hash | |
| *AbstractDataProvider | output_provider |
| the optional output data provider | |
| *AbstractDataProviderBulkOperation | output_provider_bulk_operation |
| bulk output object for an output data provider | |
| bool | output_provider_upsert |
| if the upsert operations should be used on the output provider | |
| *hash< auto > | output_request_options |
| options for the request when using the request/response API for the output provider | |
| hash< auto > | rconsth |
| map of constant runtime fields | |
| hash< string, hash< MapperRuntimeKeyInfo > > | runtime_independent_keys |
| hash of runtime keys that provide independent mappings (where there is no "requires_role" = "value") | |
| hash< string, hash< MapperRuntimeKeyInfo > > | runtime_keys |
| map of field keys provided by the "runtime_keys" option | |
| hash< string, hash< auto > > | runtime_keys_with_handler |
| subset of runtime_keys with handlers | |
| bool | structured_input |
| flag if the field descriptions were provided to the constructor More... | |
| bool | structured_output |
| flag if the field descriptions were provided to the constructor More... | |
| bool | trunc_all = False |
| truncate all strings quietly / automatically to their maximum length | |
| const | TypeMap |
| maps standard types to auto types to avoid type stripping | |
| hash< string, bool > | valid_keys |
| hash of valid keys | |
Private Member Functions | |
| AbstractDataProviderRecordIterator | getInputProviderRecordIterator () |
| returns a record iterator for an input provider More... | |
this class is a base class for mapping data; see Mapper Examples for usage examples
|
protected |
perform per-field pre-processing on the passed map in the constructor
| k | the field name |
| fh | a reference to the field's value in the map; will be converted to a hash |
|
protected |
raises an error if no output provider is present
| MAPPER-OUTPUT-BULK-ERROR | no output bulk operation is in progress |
|
protected |
raises an error if no output provider is present
| MAPPER-OUTPUT-PROVIDER-ERROR | no output provider available in this mapper |
| Mapper::Mapper::commit | ( | ) |
Commits data written to the output data provider if the output data provider supports transaction management.
Has no effect if the output data provider does not support transaction management
| MAPPER-OUTPUT-PROVIDER-ERROR | no output provider available in this mapper |
builds the object based on a hash providing field mappings, data constraints, and optionally custom mapping logic
| mapv | a hash providing field mappings; each hash key is the name of the output field; each value is either True (meaning no translations are done; the data is copied 1:1) or a hash describing the mapping; see Mapper Specification Format for detailed documentation for this option |
| opts | an optional hash of options for the mapper; see Mapper Options for a description of valid mapper options |
| MAP-ERROR | the map hash has a logical error (ex: "trunc" key given without "maxlen", invalid map key) |
| Mapper::Mapper::discardOutput | ( | ) |
Discards any buffered data in the output data provider in a bulk operation.
This method should always be called if an error occurs in a bulk output operation
| MAPPER-OUTPUT-BULK-ERROR | no output bulk operation is in progress |
|
protected |
throws a MAP-ERROR exception; prepends the map name to the description if known
if this method is subclassed, it must also cause an exception to be thrown
| Mapper::Mapper::flushOutput | ( | ) |
Flushes any remaining data to the output data provider in a bulk operation.
This method should always be called for successful bulk output operations with an output provider
| MAPPER-OUTPUT-BULK-ERROR | no output bulk operation is in progress |
| int Mapper::Mapper::getCount | ( | ) |
returns the internal record count
|
private |
returns a record iterator for an input provider
| MapperOutputRecordIterator Mapper::Mapper::getOutputIterator | ( | ) |
returns an output record iterator that produces mapped data from the input data provider
| MAPPER-INPUT-PROVIDER-ERROR | if no input_provider option was provided in the constructor or the input provider uses the request/response API |
For a possible hash of lists (bulk data) returns the size of the first list found within the hash. If no list if found, returns NOTHING.
| rec | a hash representing (possible) bulk data |
| auto Mapper::Mapper::getRuntime | ( | string | key | ) |
get current runtime option value for a key
| key | the runtime option key |
maps all input records and returns the mapped data as a list of output records
this method applies the mapData() method to all input records and returns the resulting list
| recs | the list of input records |
| MISSING-INPUT | a field marked mandatory is missing |
| STRING-TOO-LONG | a field value exceeds the maximum value and the 'trunc' key is not set |
| INVALID-NUMBER | the field is marked as numeric but the input value contains non-numeric data |
maps all input records and returns the mapped data as a list of output records
this method applies the mapData() method to all input records and returns the resulting list
| recs | a hash of lists of input records |
| MISSING-INPUT | a field marked mandatory is missing |
| STRING-TOO-LONG | a field value exceeds the maximum value and the 'trunc' key is not set |
| INVALID-NUMBER | the field is marked as numeric but the input value contains non-numeric data |
maps a set of records in hash of lists format; returns mapped data in a hash of lists format
| rec | the input record or record set in case a hash of lists is passed |
| crec | an optional simple hash of data to be added to each input row before mapping |
N rows of a column C and 'crec = {"C": "mystring"}' then the output will be as if there was 'N' rows with C = "mystring" on the input.| MISSING-INPUT | a field marked mandatory is missing |
| STRING-TOO-LONG | a field value exceeds the maximum value and the 'trunc' key is not set |
| INVALID-NUMBER | the field is marked as numeric but the input value contains non-numeric data |
processes the input record and returns a hash of the mapped values where the keys in the hash returned are the target field names; the order of the fields in the hash returned is the same order as the keys in the map hash.
| rec | the record to translate |
| MISSING-INPUT | a field marked mandatory is missing |
| STRING-TOO-LONG | a field value exceeds the maximum value and the 'trunc' key is not set |
| INVALID-NUMBER | the field is marked as numeric but the input value contains non-numeric data |
processes the input record and returns a hash of the mapped values where the keys in the hash returned are the target field names; the order of the fields in the hash returned is the same order as the keys in the map hash.
| rec | the record to translate |
| MISSING-INPUT | a field marked mandatory is missing |
| STRING-TOO-LONG | a field value exceeds the maximum value and the 'trunc' key is not set |
| INVALID-NUMBER | the field is marked as numeric but the input value contains non-numeric data |
|
protected |
maps a single field to the target
Performs the actual mapping
| h | the hash to be updated with the mapped key/value pair; Depending on the mapper specification and the input, it can contain a bulk record (hash of lists) or a hash of both list and non-list keys (these are considered a "constants" in the output record. |
| key | the column name (hash key) to be mapped (target field) |
| rec | input record - either single record of hash of lists (batch); to increase performance, the input type (single record vs. batch) is determined by the do_list parameter. In case of bulk input, all the lists are supposed to have the same length list_size |
| do_list | - whether the input record rec is single record or bulk format (hash of lists) |
| list_size | - size of the lists in case the input is in bulk format (all lists must have the same length, asserted inside the method). |
| hash<auto> Mapper::Mapper::optionKeys | ( | ) |
returns a list of valid constructor options for this class (can be overridden in subclasses)
| Mapper::Mapper::replaceRuntime | ( | *hash< auto > | runtime | ) |
replaces runtime options
| runtime | a hash of runtime options to use to replace the current runtime option hash |
| RUNTIME-OPTION-CHANGED | the option has changed during while already processing the data. |
| Mapper::Mapper::resetCount | ( | ) |
resets the internal record count
| Mapper::Mapper::rollback | ( | ) |
Rolls back data written to the output data provider.
Has no effect if the output data provider does not support transaction management
| MAPPER-OUTPUT-PROVIDER-ERROR | no output provider available in this mapper |
| Mapper::Mapper::runAutonomous | ( | ) |
Runs the input and output mappers with data providers on each end autonomously.
| MAPPER-INPUT-PROVIDER-ERROR | no input provider available in this mapper |
| MAPPER-OUTPUT-PROVIDER-ERROR | no output provider available in this mapper |
| Mapper::Mapper::setRuntime | ( | string | key, |
| auto | value | ||
| ) |
set the runtime option with "key" to value "value"
| key | a string with valid runtime key |
| value | anything passed to the current runtime key |
| RUNTIME-OPTION-CHANGED | the option has changed during while already processing the data. |
| Mapper::Mapper::setRuntime | ( | hash< auto > | runtime | ) |
adds runtime options to the current runtime option hash
| runtime | a hash of runtime options to add to the current runtime option hash |
| RUNTIME-OPTION-CHANGED | the option has changed during while already processing the data. |
returns a list of valid field keys for this class (can be overridden in subclasses)
|
protected |
DEPRECATED: do not assume struct when field names have a "." in them; instead allow input field names to have a "." in them.
only used when a plain hash is provided for input
|
protected |
DEPRECATED: do not assume structured/hash output when output field names have a "." in them; instead allow output field names to have a "." in them.
only used when a plain hash is provided for output
|
protected |
the hash providing output field names and mappings
note that we must strip types here
|
protected |
flag if the field descriptions were provided to the constructor
if so then allow_dot cannot be set
|
protected |
flag if the field descriptions were provided to the constructor
if so then allow_output_dot cannot be set