The entity exporters and importers processors API, currently part of the Scipio Data Connect API, provides a framework of java classes and JSON-friendly services for writing, extending and remotely invoking entity exporters/importers for product, party, order, customer and other data. These help interconnect remote systems and applications and simplify data handling.
Several export services named entityExport* (e.g. entityExportProduct, entityExportOrder) are provided that produce a JSON-like record for each main entity read (some exporters accumulate multiple entities per produced record). By default these are returned as a list of records as the “records” OUT attribute. Alongside these are entityImport* import services that create and update entities from JSON-like records from a “records” IN attribute, which usually closely matches the export format (not guaranteed due to custom fields and needs).
These services are implemented using EntityExporter/EntityImporter base classes. All exporters and importers and can customized using the file: entityprocessors.properties (currently within addons/data-connect/config/). This file can be defined in the config folder of any component, or the existing one can be edited in place.
Record Format
Entity exporters produce one JSON-like record per related entity, logical primary key or sequential entity values, and by default return a list of records. Typically a record’s top map (JSON-like) is based on one main entity or view-entity definition, and then extra fields can be added (lowercase-starting key names) and finally more entity records known as “entity keys” whose values are lists of sub-records reflecting entities related to the main record’s logical entity (uppercase-starting key names, EntityName and EntityName-Title). As returned by entityExportCatalog on stock Scipio data:
[
{
"prodCatalogId": "DemoCatalog",
"catalogName": "Demo Catalog",
"useQuickAdd": "Y",
"styleSheet": null,
"headerLogo": null,
"contentPathPrefix": null,
"templatePathPrefix": null,
"viewAllowPermReqd": null,
"purchaseAllowPermReqd": null,
"_entityName": "ProdCatalog",
"ProdCatalogCategory": [
{
"prodCatalogId": "DemoCatalog",
"productCategoryId": "CATALOG1",
"prodCatalogCategoryTypeId": "PCCT_BROWSE_ROOT",
"fromDate": 989769600000,
"thruDate": null,
"sequenceNum": 1,
"_entityName": "ProdCatalogCategory"
},
{
"prodCatalogId": "DemoCatalog",
"productCategoryId": "PROMOTIONS",
"prodCatalogCategoryTypeId": "PCCT_PROMOTIONS",
"fromDate": 989769600000,
"thruDate": null,
"sequenceNum": 10,
"_entityName": "ProdCatalogCategory"
},
// ...
{
"prodCatalogId": "DemoCatalog",
"productCategoryId": "CATALOG1_QUICKADD2",
"prodCatalogCategoryTypeId": "PCCT_QUICK_ADD",
"fromDate": 989769600000,
"thruDate": null,
"sequenceNum": 62,
"_entityName": "ProdCatalogCategory"
}
],
"ProductStoreCatalog": [
{
"productStoreId": "ScipioShop",
"prodCatalogId": "DemoCatalog",
"fromDate": 989769600000,
"thruDate": null,
"sequenceNum": 1,
"_entityName": "ProductStoreCatalog"
}
]
},
{
"prodCatalogId": "TestCatalog",
"catalogName": "Test Catalog",
"useQuickAdd": "N",
"styleSheet": null,
"headerLogo": null,
"contentPathPrefix": null,
"templatePathPrefix": null,
"viewAllowPermReqd": null,
"purchaseAllowPermReqd": null,
"_entityName": "ProdCatalog",
"ProdCatalogCategory": [
{
"prodCatalogId": "TestCatalog",
"productCategoryId": "ELTRN-100",
"prodCatalogCategoryTypeId": "PCCT_BROWSE_ROOT",
"fromDate": 989769600000,
"thruDate": null,
"sequenceNum": 1,
"_entityName": "ProdCatalogCategory"
},
{
"prodCatalogId": "TestCatalog",
"productCategoryId": "TSTLTDADMIN",
"prodCatalogCategoryTypeId": "PCCT_ADMIN_ALLW",
"fromDate": 989769600000,
"thruDate": null,
"sequenceNum": 20,
"_entityName": "ProdCatalogCategory"
}
],
"ProductStoreCatalog": [
{
"productStoreId": "ScipioShop",
"prodCatalogId": "TestCatalog",
"fromDate": 989769600000,
"thruDate": null,
"sequenceNum": 2,
"_entityName": "ProductStoreCatalog"
}
]
},
// ...
]
When comes time to import, entity keys are read back and automatically create or update their corresponding entities in the database. A special key “_entityName” is consulted and takes precedence over the entity key when determining the entity name.
Entity imports with uppercase entity keys also happens recursively, such that if the top record contains a list of InventoryItem entities (see ProductInventory exporter/importer), each of these can contain InventoryItemDetail lists. InventoryItemDetail has a many-to-one relation with InventoryItem and this is reflected by the detail being child of the item. In other words, entity keys are consulted recursively for entities to import at every record and sub-record map unless prevented by importer (or omitted by exporter). This recursion and logic is not automatically applied to other record keys that aren’t (uppercase-started) entity keys, but exporters/importers are free to handle custom keys any way they want.
Special case and ordering: The record itself is always imported before the entities from its entity keys, except in one case: if the entity key is the very first key(s) in the record map before any non-uppercased map key appears. It is assumed all records have insertion-order-preserving keys and ordering such that all classes in the API use Java’s LinkedHashMap to back records.
Configuration and extension
entityprocessors.properties defines the settings, entities to skip and factory classes to use for each of the exporters and importers.
By default, a significant number of entities are exported by entityExportProduct. To limit its default exporter to make it smaller and faster, one could ignore some of the exported entities:
product.export.entityKey.ignore.names=ProductManufacturingRule,ProductManufacturingRule-ProductFor
However, a typical client project will want to extend the factory and override or augment its behavior:
product.export.factoryClass=com.ilscipio.scipio.data.ProductExporter$Factory
Several methods may be overridden as examplified in the provided exporters such as com.ilscipio.scipio.data.ProductExporter and those under the same package.
Export API
Entity exporters extend EntityExporter and implement the entityExportInterface service interface, are named entityExport* and are export=”true”:
<service name="entityExportInterface" engine="interface">
<description>Entity export to json-like record common interface for DataConnect services</description>
<attribute name="entities" type="Object" mode="IN" optional="true">
<description>Optional list of entities or primary keys to export, otherwise all; may be iterator or collection</description>
</attribute>
<attribute name="entityFilters" type="List" mode="IN" optional="true">
<description>Optional entity filters, instances of com.ilscipio.scipio.data.EntityExporter$EntityFilter</description>
</attribute>
<attribute name="recordsFormat" type="Object" mode="IN" optional="true" default-value="List">
<description>Determines the format of the returned records, one of:
com.ilscipio.scipio.data.EntityExporter$Collector - record collector instance (may return list, map or neither/other),
"List" or List instance - list of records,
"Map" or Map - map of record keys (logical primary key) to records</description>
</attribute>
<attribute name="filterByDate" type="Boolean" mode="IN" optional="true" default-value="false">
<description>Filter by date flag</description>
</attribute>
<attribute name="includeNull" type="Boolean" mode="IN" optional="true" default-value="true">
<description>If false, null values omitted from output (not recommended for re-import or certain formats)</description>
</attribute>
<attribute name="useCache" type="Boolean" mode="IN" optional="true" default-value="false">
<description>useCache, not recommended</description>
</attribute>
<attribute name="bufSize" type="Integer" mode="IN" optional="true">
<description>Buffer size, otherwise configuration default (dataconnect.properties)</description>
</attribute>
<attribute name="records" type="Object" mode="OUT" optional="true">
<description>Results record list or map as determined by recordsFormat or as provided by the record collector passed in it (collectors may return null)</description>
</attribute>
</service>
The exported services require admin rights. Remotely with HTTPS, the standard service attributes “login.username”/”login.password” can be used to authenticate. An alternative set of non-exported services, entityExport*Internal without security checks are provided for situations where security is already provided.
By default, each export service finds all records in the system and returns them in the “records” OUT attribute, but these can be limited using the entities and entityFilters attributes, which is essential for large databases. Note: Large databases must use entity list or filters to limit the amount of data.
Records are returned as a List of records in the “records” OUT attribute, or as a Map of record values keyed by short primary key if recordsFormat is Map. If recordsFormat is an instance of EntityExporter$Collector, the records can be handled a custom way as they are generated. The format of the individual records is dependent on the exporter and data type but is usually almost the same for an exporter/importer pair.
Being export=”true” services they are published using WSDL/SOAP for remote communication (https://localhost:8443/admin/control/ServiceList?sel_service_name=entityExportCatalog&show_wsdl=true). For modern connections, the Data Connect webapp hosts HTTPS/JSON requests under the name: /dataconnect/control/entityExport*. Example: Using the command line tool curl, send a JSON export request (Note: Do not use the default admin account outside of testing and a different account should be used on production):
curl https://localhost:8443/dataconnect/control/entityExportParty --insecure --header "Content-Type: application/json" --request POST --data '
{
"roleTypeId" : "INTERNAL_ORGANIZATIO",
"login.username" : "admin",
"login.password" : "scipio",
"whereCond": {"partyId" : "Company"},
"orderBy": ["partyId"]
}'
entityExportCatalog
Main record entity: ProdCatalog
{
"prodCatalogId": "DemoCatalog",
"catalogName": "Demo Catalog",
"useQuickAdd": "Y",
"styleSheet": null,
"headerLogo": null,
"contentPathPrefix": null,
"templatePathPrefix": null,
"viewAllowPermReqd": null,
"purchaseAllowPermReqd": null,
"_entityName": "ProdCatalog",
"ProdCatalogCategory": [
{
"prodCatalogId": "DemoCatalog",
"productCategoryId": "CATALOG1",
"prodCatalogCategoryTypeId": "PCCT_BROWSE_ROOT",
"fromDate": 989769600000,
"thruDate": null,
"sequenceNum": 1,
"_entityName": "ProdCatalogCategory"
},
{
"prodCatalogId": "DemoCatalog",
"productCategoryId": "PROMOTIONS",
"prodCatalogCategoryTypeId": "PCCT_PROMOTIONS",
"fromDate": 989769600000,
"thruDate": null,
"sequenceNum": 10,
"_entityName": "ProdCatalogCategory"
},
// ...
{
"prodCatalogId": "DemoCatalog",
"productCategoryId": "CATALOG1_QUICKADD2",
"prodCatalogCategoryTypeId": "PCCT_QUICK_ADD",
"fromDate": 989769600000,
"thruDate": null,
"sequenceNum": 62,
"_entityName": "ProdCatalogCategory"
}
],
"ProductStoreCatalog": [
{
"productStoreId": "ScipioShop",
"prodCatalogId": "DemoCatalog",
"fromDate": 989769600000,
"thruDate": null,
"sequenceNum": 1,
"_entityName": "ProductStoreCatalog"
}
]
}
entityExportCategory
Main record entity: ProductCategory
{
"productCategoryId": "PC-PR-101",
"productCategoryTypeId": "CATALOG_CATEGORY",
"primaryParentCategoryId": "PC-AC-101",
"categoryName": null,
"description": "Peripherals",
"longDescription": "Peripherals",
"categoryImageUrl": null,
"linkOneImageUrl": null,
"linkTwoImageUrl": null,
"detailScreen": null,
"showInSelect": null,
"_entityName": "ProductCategory",
"trail": "4/CATALOG1/ELTRN-100/PC-101/PC-AC-101/PC-PR-101",
"categoryRollups": [
[
"CATALOG1",
"ELTRN-100",
"PC-101",
"PC-AC-101",
"PC-PR-101"
]
],
"ProdCatalogCategory": null,
"ProductCategoryAttribute": null,
"ProductCategoryContent": [
{
"productCategoryId": "PC-PR-101",
"contentId": "PC-PR-101-ALTEN",
"prodCatContentTypeId": "ALTERNATIVE_URL",
"fromDate": 1513135635392,
"thruDate": null,
"purchaseFromDate": null,
"purchaseThruDate": null,
"useCountLimit": null,
"useDaysLimit": null,
"_entityName": "ProductCategoryContent"
},
{
"productCategoryId": "PC-PR-101",
"contentId": "CPC-PR-101DESCEN",
"prodCatContentTypeId": "CATEGORY_NAME",
"fromDate": 1158897600000,
"thruDate": null,
"purchaseFromDate": null,
"purchaseThruDate": null,
"useCountLimit": null,
"useDaysLimit": null,
"_entityName": "ProductCategoryContent"
}
],
"ProductCategoryGlAccount": null,
"ProductCategoryLink": null,
"ProductCategoryRole": null,
"ProductCategoryRollup-Children": null,
"ProductCategoryRollup-Parents": [
{
"productCategoryId": "PC-PR-101",
"parentProductCategoryId": "PC-AC-101",
"fromDate": 989769600000,
"thruDate": null,
"sequenceNum": null,
"_entityName": "ProductCategoryRollup"
}
],
"ProductFeatureCategoryAppl": null,
"ProductFeatureCatGrpAppl": null
}
entityExportProduct
Main record entity: Product
The product export follows the same format as the other exporters but with significantly more fields and entities, derived from the entity schema and the product . It is recommended to filter out unwanted using entityprocessors.properties or customizations as the number of queries can end up very high.
entityExportProductInventory
Main record entity: (none – only productId)
{
"productId": "CAM-2644",
"InventoryItem": [
{
"inventoryItemId": "9000",
"inventoryItemTypeId": "NON_SERIAL_INV_ITEM",
"productId": "CAM-2644",
"partyId": null,
"ownerPartyId": "Company",
"statusId": null,
"datetimeReceived": 1217592000000,
"datetimeManufactured": null,
"expireDate": null,
"facilityId": "ScipioShopWarehouse",
"containerId": null,
"lotId": null,
"uomId": null,
"binNumber": null,
"locationSeqId": "TLTLTLUL01",
"comments": null,
"quantityOnHandTotal": 500.000000,
"availableToPromiseTotal": 500.000000,
"accountingQuantityTotal": 500.000000,
"oldQuantityOnHand": null,
"oldAvailableToPromise": null,
"serialNumber": null,
"softIdentifier": null,
"activationNumber": null,
"activationValidThru": null,
"unitCost": 2.500000,
"currencyUomId": "USD",
"fixedAssetId": null,
"_entityName": "InventoryItem",
"InventoryItemDetail": [
{
"workEffortId": null,
"orderId": null,
"createdStamp": 1600461844758,
"description": null,
"maintHistSeqId": null,
"inventoryItemDetailSeqId": "0001",
"accountingQuantityDiff": 500.000000,
"physicalInventoryId": null,
"itemIssuanceId": null,
"returnId": null,
"reasonEnumId": null,
"receiptId": null,
"quantityOnHandDiff": 500.000000,
"orderItemSeqId": null,
"lastUpdatedStamp": 1600461844758,
"createdTxStamp": 1600461841476,
"lastUpdatedTxStamp": 1600461841476,
"shipmentItemSeqId": null,
"shipGroupSeqId": null,
"inventoryItemId": "9000",
"availableToPromiseDiff": 500.000000,
"shipmentId": null,
"unitCost": null,
"returnItemSeqId": null,
"fixedAssetId": null,
"effectiveDate": 989769600000
}
]
},
{
"inventoryItemId": "9001",
"inventoryItemTypeId": "NON_SERIAL_INV_ITEM",
"productId": "CAM-2644",
"partyId": null,
"ownerPartyId": "Company",
"statusId": null,
"datetimeReceived": 1217592000000,
"datetimeManufactured": null,
"expireDate": null,
"facilityId": "ScipioShopWarehouse",
"containerId": null,
"lotId": null,
"uomId": null,
"binNumber": null,
"locationSeqId": "TLTLTLLL01",
"comments": null,
"quantityOnHandTotal": 5.000000,
"availableToPromiseTotal": 5.000000,
"accountingQuantityTotal": 5.000000,
"oldQuantityOnHand": null,
"oldAvailableToPromise": null,
"serialNumber": null,
"softIdentifier": null,
"activationNumber": null,
"activationValidThru": null,
"unitCost": 2.500000,
"currencyUomId": "USD",
"fixedAssetId": null,
"_entityName": "InventoryItem",
"InventoryItemDetail": [
{
"workEffortId": null,
"orderId": null,
"createdStamp": 1600461845110,
"description": null,
"maintHistSeqId": null,
"inventoryItemDetailSeqId": "0001",
"accountingQuantityDiff": 5.000000,
"physicalInventoryId": null,
"itemIssuanceId": null,
"returnId": null,
"reasonEnumId": null,
"receiptId": null,
"quantityOnHandDiff": 5.000000,
"orderItemSeqId": null,
"lastUpdatedStamp": 1600461845110,
"createdTxStamp": 1600461841476,
"lastUpdatedTxStamp": 1600461841476,
"shipmentItemSeqId": null,
"shipGroupSeqId": null,
"inventoryItemId": "9001",
"availableToPromiseDiff": 5.000000,
"shipmentId": null,
"unitCost": null,
"returnItemSeqId": null,
"fixedAssetId": null,
"effectiveDate": 989769600000
}
]
},
{
"inventoryItemId": "9025",
"inventoryItemTypeId": "NON_SERIAL_INV_ITEM",
"productId": "CAM-2644",
"partyId": null,
"ownerPartyId": "Company",
"statusId": null,
"datetimeReceived": 1250200051095,
"datetimeManufactured": null,
"expireDate": null,
"facilityId": "ScipioShopWarehouse",
"containerId": null,
"lotId": null,
"uomId": null,
"binNumber": null,
"locationSeqId": "TLTLTLLL01",
"comments": null,
"quantityOnHandTotal": 4.000000,
"availableToPromiseTotal": 4.000000,
"accountingQuantityTotal": 2.000000,
"oldQuantityOnHand": null,
"oldAvailableToPromise": null,
"serialNumber": null,
"softIdentifier": null,
"activationNumber": null,
"activationValidThru": null,
"unitCost": 24.000000,
"currencyUomId": "USD",
"fixedAssetId": null,
"_entityName": "InventoryItem",
"InventoryItemDetail": [
{
"workEffortId": null,
"orderId": "Demo1001",
"createdStamp": 1600461867218,
"description": null,
"maintHistSeqId": null,
"inventoryItemDetailSeqId": "00001",
"accountingQuantityDiff": 2.000000,
"physicalInventoryId": null,
"itemIssuanceId": null,
"returnId": null,
"reasonEnumId": null,
"receiptId": "9000",
"quantityOnHandDiff": 2.000000,
"orderItemSeqId": "00001",
"lastUpdatedStamp": 1600461867218,
"createdTxStamp": 1600461865028,
"lastUpdatedTxStamp": 1600461865028,
"shipmentItemSeqId": null,
"shipGroupSeqId": null,
"inventoryItemId": "9025",
"availableToPromiseDiff": 2.000000,
"shipmentId": "9997",
"unitCost": 24.000000,
"returnItemSeqId": null,
"fixedAssetId": null,
"effectiveDate": 1250200066780
},
{
"workEffortId": null,
"orderId": null,
"createdStamp": 1600461867000,
"description": null,
"maintHistSeqId": null,
"inventoryItemDetailSeqId": "10000",
"accountingQuantityDiff": 0.000000,
"physicalInventoryId": null,
"itemIssuanceId": null,
"returnId": null,
"reasonEnumId": null,
"receiptId": null,
"quantityOnHandDiff": 2.000000,
"orderItemSeqId": null,
"lastUpdatedStamp": 1600461867000,
"createdTxStamp": 1600461865028,
"lastUpdatedTxStamp": 1600461865028,
"shipmentItemSeqId": null,
"shipGroupSeqId": null,
"inventoryItemId": "9025",
"availableToPromiseDiff": 2.000000,
"shipmentId": null,
"unitCost": null,
"returnItemSeqId": null,
"fixedAssetId": null,
"effectiveDate": 1600461867000
}
]
}
]
}
entityExportParty
Main record entity: PartyAndRole (Party if no roleTypeId)
{
"partyId": "DemoEmployee",
"partyTypeId": null,
"externalId": null,
"preferredCurrencyUomId": null,
"description": null,
"statusId": null,
"createdDate": null,
"createdByUserLogin": null,
"lastModifiedDate": null,
"lastModifiedByUserLogin": null,
"dataSourceId": null,
"isUnread": null,
"_entityName": "Party",
"Party": [
{
"partyId": "DemoEmployee",
"partyTypeId": null,
"externalId": null,
"preferredCurrencyUomId": null,
"description": null,
"statusId": null,
"createdDate": null,
"createdByUserLogin": null,
"lastModifiedDate": null,
"lastModifiedByUserLogin": null,
"dataSourceId": null,
"isUnread": null,
"_entityName": "Party"
}
],
"PartyRole": [
{
"partyId": "DemoEmployee",
"roleTypeId": "EMAIL_ADMIN",
"_entityName": "PartyRole"
},
{
"partyId": "DemoEmployee",
"roleTypeId": "EMPLOYEE",
"_entityName": "PartyRole"
},
{
"partyId": "DemoEmployee",
"roleTypeId": "REQ_TAKER",
"_entityName": "PartyRole"
}
],
"UserLogin": [
{
"userLoginId": "demoemployee",
"currentPassword": "{SHA}4afca3ceb9bb4b053efaf77766b5323163bd2266",
"passwordHint": null,
"isSystem": null,
"enabled": null,
"hasLoggedOut": null,
"requirePasswordChange": "N",
"lastCurrencyUom": null,
"lastLocale": null,
"lastTimeZone": null,
"disabledDateTime": null,
"successiveFailedLogins": null,
"externalAuthId": null,
"userLdapDn": null,
"disabledBy": null,
"partyId": "DemoEmployee",
"_entityName": "UserLogin"
}
],
"Person": null
}
entityExportCompany
Main record entity: PartyAndGroupAndRole (roleTypeId INTERNAL_ORGANIZATIO)
{
"partyId": "Company",
"partyTypeId": "PARTY_GROUP",
"externalId": null,
"preferredCurrencyUomId": null,
"description": null,
"statusId": null,
"createdDate": null,
"createdByUserLogin": null,
"lastModifiedDate": null,
"lastModifiedByUserLogin": null,
"dataSourceId": null,
"isUnread": null,
"roleTypeId": "INTERNAL_ORGANIZATIO",
"_entityName": "PartyAndRole",
"contactMechsByPurpose": {
"BILLING_LOCATION": [
{
"purposeFromDate": 946702800000,
"purposeThruDate": null,
"partyId": "Company",
"contactMechId": "9000",
"fromDate": 946702800000,
"thruDate": null,
"roleTypeId": null,
"allowSolicitation": "Y",
"extension": null,
"verified": null,
"comments": null,
"yearsWithContactMech": null,
"monthsWithContactMech": null,
"contactMechTypeId": "POSTAL_ADDRESS",
"infoString": null,
"toName": "Company XYZ",
"attnName": null,
"address1": "2003 Open Blvd",
"address2": null,
"directions": null,
"city": "Open City",
"postalCode": "999999",
"postalCodeExt": null,
"countryGeoId": "USA",
"stateProvinceGeoId": "CA",
"countyGeoId": null,
"postalCodeGeoId": null,
"geoPointId": "9000",
"contactMechPurposeTypeId": "BILLING_LOCATION",
"_entityName": "PartyContactMechAndPostalAddressAndPurpose"
}
],
"GENERAL_LOCATION": [
{
"purposeFromDate": 946702800000,
"purposeThruDate": null,
"partyId": "Company",
"contactMechId": "9000",
"fromDate": 946702800000,
"thruDate": null,
"roleTypeId": null,
"allowSolicitation": "Y",
"extension": null,
"verified": null,
"comments": null,
"yearsWithContactMech": null,
"monthsWithContactMech": null,
"contactMechTypeId": "POSTAL_ADDRESS",
"infoString": null,
"toName": "Company XYZ",
"attnName": null,
"address1": "2003 Open Blvd",
"address2": null,
"directions": null,
"city": "Open City",
"postalCode": "999999",
"postalCodeExt": null,
"countryGeoId": "USA",
"stateProvinceGeoId": "CA",
"countyGeoId": null,
"postalCodeGeoId": null,
"geoPointId": "9000",
"contactMechPurposeTypeId": "GENERAL_LOCATION",
"_entityName": "PartyContactMechAndPostalAddressAndPurpose"
}
],
"PAYMENT_LOCATION": [
{
"purposeFromDate": 946702800000,
"purposeThruDate": null,
"partyId": "Company",
"contactMechId": "9000",
"fromDate": 946702800000,
"thruDate": null,
"roleTypeId": null,
"allowSolicitation": "Y",
"extension": null,
"verified": null,
"comments": null,
"yearsWithContactMech": null,
"monthsWithContactMech": null,
"contactMechTypeId": "POSTAL_ADDRESS",
"infoString": null,
"toName": "Company XYZ",
"attnName": null,
"address1": "2003 Open Blvd",
"address2": null,
"directions": null,
"city": "Open City",
"postalCode": "999999",
"postalCodeExt": null,
"countryGeoId": "USA",
"stateProvinceGeoId": "CA",
"countyGeoId": null,
"postalCodeGeoId": null,
"geoPointId": "9000",
"contactMechPurposeTypeId": "PAYMENT_LOCATION",
"_entityName": "PartyContactMechAndPostalAddressAndPurpose"
}
]
},
"Party": [
{
"partyId": "Company",
"partyTypeId": "PARTY_GROUP",
"externalId": null,
"preferredCurrencyUomId": null,
"description": null,
"statusId": null,
"createdDate": null,
"createdByUserLogin": null,
"lastModifiedDate": null,
"lastModifiedByUserLogin": null,
"dataSourceId": null,
"isUnread": null,
"_entityName": "Party"
}
],
"PartyRole": [
{
"partyId": "Company",
"roleTypeId": "ACCOUNT",
"_entityName": "PartyRole"
},
{
"partyId": "Company",
"roleTypeId": "BILL_FROM_VENDOR",
"_entityName": "PartyRole"
},
{
"partyId": "Company",
"roleTypeId": "BILL_TO_CUSTOMER",
"_entityName": "PartyRole"
},
{
"partyId": "Company",
"roleTypeId": "CARRIER",
"_entityName": "PartyRole"
},
{
"partyId": "Company",
"roleTypeId": "INTERNAL_ORGANIZATIO",
"_entityName": "PartyRole"
},
{
"partyId": "Company",
"roleTypeId": "_NA_",
"_entityName": "PartyRole"
}
],
"UserLogin": null,
"Person": null
}
entityExportCustomer
Main record entity: PartyAndUserLoginAndPersonAndRoleFull (roleTypeId CUSTOMER)
{
"partyId": "EuroCustomer",
"partyTypeId": "PERSON",
"externalId": null,
"preferredCurrencyUomId": "EUR",
"description": null,
"statusId": "PARTY_ENABLED",
"createdDate": null,
"createdByUserLogin": null,
"lastModifiedDate": null,
"lastModifiedByUserLogin": null,
"dataSourceId": null,
"isUnread": null,
"userLoginId": "EuroCustomer",
"currentPassword": null,
"passwordHint": null,
"isSystem": null,
"enabled": null,
"hasLoggedOut": null,
"requirePasswordChange": null,
"lastCurrencyUom": null,
"lastLocale": null,
"lastTimeZone": null,
"disabledDateTime": null,
"successiveFailedLogins": null,
"externalAuthId": null,
"userLdapDn": null,
"disabledBy": null,
"salutation": null,
"firstName": "Euro",
"middleName": null,
"lastName": "Customer",
"personalTitle": null,
"suffix": null,
"nickname": null,
"firstNameLocal": null,
"middleNameLocal": null,
"lastNameLocal": null,
"otherLocal": null,
"memberId": null,
"gender": null,
"birthDate": null,
"deceasedDate": null,
"height": null,
"weight": null,
"mothersMaidenName": null,
"maritalStatus": null,
"socialSecurityNumber": null,
"passportNumber": null,
"passportExpireDate": null,
"totalYearsWorkExperience": null,
"comments": null,
"employmentStatusEnumId": null,
"residenceStatusEnumId": null,
"occupation": null,
"yearsWithEmployer": null,
"monthsWithEmployer": null,
"existingCustomer": null,
"cardId": null,
"roleTypeId": "CUSTOMER",
"_entityName": "PartyAndUserLoginAndPersonAndRoleFull",
"contactMechsByPurpose": {
"BILLING_LOCATION": [
{
"purposeFromDate": 989726400000,
"purposeThruDate": null,
"partyId": "EuroCustomer",
"contactMechId": "EUROCUSTOMER",
"fromDate": 989726400000,
"thruDate": null,
"roleTypeId": null,
"allowSolicitation": "Y",
"extension": null,
"verified": null,
"comments": null,
"yearsWithContactMech": null,
"monthsWithContactMech": null,
"contactMechTypeId": "POSTAL_ADDRESS",
"infoString": null,
"toName": "Euro Customer",
"attnName": null,
"address1": "Prinsengracht 3",
"address2": null,
"directions": null,
"city": "Amsterdam",
"postalCode": "1000BD",
"postalCodeExt": null,
"countryGeoId": "NLD",
"stateProvinceGeoId": null,
"countyGeoId": null,
"postalCodeGeoId": null,
"geoPointId": "9001",
"contactMechPurposeTypeId": "BILLING_LOCATION",
"_entityName": "PartyContactMechAndPostalAddressAndPurpose"
}
],
"GENERAL_LOCATION": [
{
"purposeFromDate": 989726400000,
"purposeThruDate": null,
"partyId": "EuroCustomer",
"contactMechId": "EUROCUSTOMER",
"fromDate": 989726400000,
"thruDate": null,
"roleTypeId": null,
"allowSolicitation": "Y",
"extension": null,
"verified": null,
"comments": null,
"yearsWithContactMech": null,
"monthsWithContactMech": null,
"contactMechTypeId": "POSTAL_ADDRESS",
"infoString": null,
"toName": "Euro Customer",
"attnName": null,
"address1": "Prinsengracht 3",
"address2": null,
"directions": null,
"city": "Amsterdam",
"postalCode": "1000BD",
"postalCodeExt": null,
"countryGeoId": "NLD",
"stateProvinceGeoId": null,
"countyGeoId": null,
"postalCodeGeoId": null,
"geoPointId": "9001",
"contactMechPurposeTypeId": "GENERAL_LOCATION",
"_entityName": "PartyContactMechAndPostalAddressAndPurpose"
}
],
"SHIPPING_LOCATION": [
{
"purposeFromDate": 989726400000,
"purposeThruDate": null,
"partyId": "EuroCustomer",
"contactMechId": "EUROCUSTOMER",
"fromDate": 989726400000,
"thruDate": null,
"roleTypeId": null,
"allowSolicitation": "Y",
"extension": null,
"verified": null,
"comments": null,
"yearsWithContactMech": null,
"monthsWithContactMech": null,
"contactMechTypeId": "POSTAL_ADDRESS",
"infoString": null,
"toName": "Euro Customer",
"attnName": null,
"address1": "Prinsengracht 3",
"address2": null,
"directions": null,
"city": "Amsterdam",
"postalCode": "1000BD",
"postalCodeExt": null,
"countryGeoId": "NLD",
"stateProvinceGeoId": null,
"countyGeoId": null,
"postalCodeGeoId": null,
"geoPointId": "9001",
"contactMechPurposeTypeId": "SHIPPING_LOCATION",
"_entityName": "PartyContactMechAndPostalAddressAndPurpose"
}
]
},
"Party": [
{
"partyId": "EuroCustomer",
"partyTypeId": "PERSON",
"externalId": null,
"preferredCurrencyUomId": "EUR",
"description": null,
"statusId": "PARTY_ENABLED",
"createdDate": null,
"createdByUserLogin": null,
"lastModifiedDate": null,
"lastModifiedByUserLogin": null,
"dataSourceId": null,
"isUnread": null,
"_entityName": "Party"
}
],
"PartyRole": [
{
"partyId": "EuroCustomer",
"roleTypeId": "BILL_TO_CUSTOMER",
"_entityName": "PartyRole"
},
{
"partyId": "EuroCustomer",
"roleTypeId": "CUSTOMER",
"_entityName": "PartyRole"
}
],
"UserLogin": [
{
"userLoginId": "EuroCustomer",
"currentPassword": null,
"passwordHint": null,
"isSystem": null,
"enabled": null,
"hasLoggedOut": null,
"requirePasswordChange": null,
"lastCurrencyUom": null,
"lastLocale": null,
"lastTimeZone": null,
"disabledDateTime": null,
"successiveFailedLogins": null,
"externalAuthId": null,
"userLdapDn": null,
"disabledBy": null,
"partyId": "EuroCustomer",
"_entityName": "UserLogin"
}
],
"Person": [
{
"partyId": "EuroCustomer",
"salutation": null,
"firstName": "Euro",
"middleName": null,
"lastName": "Customer",
"personalTitle": null,
"suffix": null,
"nickname": null,
"firstNameLocal": null,
"middleNameLocal": null,
"lastNameLocal": null,
"otherLocal": null,
"memberId": null,
"gender": null,
"birthDate": null,
"deceasedDate": null,
"height": null,
"weight": null,
"mothersMaidenName": null,
"maritalStatus": null,
"socialSecurityNumber": null,
"passportNumber": null,
"passportExpireDate": null,
"totalYearsWorkExperience": null,
"comments": null,
"employmentStatusEnumId": null,
"residenceStatusEnumId": null,
"occupation": null,
"yearsWithEmployer": null,
"monthsWithEmployer": null,
"existingCustomer": null,
"cardId": null,
"_entityName": "Person"
}
]
}
entityExportOrder
Main record entity: OrderHeader
{
"orderId": "DEMO10090",
"orderTypeId": "SALES_ORDER",
"orderName": null,
"externalId": null,
"salesChannelEnumId": "WEB_SALES_CHANNEL",
"orderDate": 1208983767392,
"priority": "2",
"entryDate": 1208983767392,
"pickSheetPrintedDate": null,
"visitId": "10002",
"statusId": "ORDER_APPROVED",
"createdBy": "admin",
"firstAttemptOrderId": null,
"currencyUom": "USD",
"syncStatusId": null,
"billingAccountId": null,
"originFacilityId": null,
"webSiteId": null,
"productStoreId": "ScipioShop",
"terminalId": null,
"transactionId": null,
"autoOrderShoppingListId": null,
"needsInventoryIssuance": null,
"isRushOrder": null,
"internalCode": null,
"remainingSubTotal": 38.40,
"grandTotal": 134.03,
"isViewed": null,
"invoicePerShipment": "Y",
"_entityName": "OrderHeader",
"orderSubTotal": 121.58,
"otherAdjAmount": 0.00,
"shippingAmount": 12.45,
"taxAmount": 0,
"orderVATTaxTotal": 0,
"distributorId": null,
"affiliateId": null,
"OrderAdjustment": [
{
"orderAdjustmentId": "9000",
"orderAdjustmentTypeId": "PROMOTION_ADJUSTMENT",
"orderId": "DEMO10090",
"orderItemSeqId": "00001",
"shipGroupSeqId": "_NA_",
"comments": null,
"description": null,
"amount": -38.400,
"recurringAmount": null,
"amountAlreadyIncluded": null,
"productPromoId": "9016",
"productPromoRuleId": "01",
"productPromoActionSeqId": "01",
"productFeatureId": null,
"correspondingProductId": null,
"taxAuthorityRateSeqId": null,
"sourceReferenceId": null,
"sourcePercentage": null,
"customerReferenceId": null,
"primaryGeoId": null,
"secondaryGeoId": null,
"exemptAmount": null,
"taxAuthGeoId": null,
"taxAuthPartyId": null,
"overrideGlAccountId": null,
"includeInTax": null,
"includeInShipping": null,
"isManual": null,
"createdDate": 1208983767866,
"createdByUserLogin": "admin",
"lastModifiedDate": null,
"lastModifiedByUserLogin": null,
"originalAdjustmentId": null,
"oldAmountPerQuantity": null,
"oldPercentage": null,
"_entityName": "OrderAdjustment"
},
{
"orderAdjustmentId": "9001",
"orderAdjustmentTypeId": "SHIPPING_CHARGES",
"orderId": "DEMO10090",
"orderItemSeqId": "_NA_",
"shipGroupSeqId": "00001",
"comments": null,
"description": null,
"amount": 12.450,
"recurringAmount": null,
"amountAlreadyIncluded": null,
"productPromoId": null,
"productPromoRuleId": null,
"productPromoActionSeqId": null,
"productFeatureId": null,
"correspondingProductId": null,
"taxAuthorityRateSeqId": null,
"sourceReferenceId": null,
"sourcePercentage": null,
"customerReferenceId": null,
"primaryGeoId": null,
"secondaryGeoId": null,
"exemptAmount": null,
"taxAuthGeoId": null,
"taxAuthPartyId": null,
"overrideGlAccountId": null,
"includeInTax": null,
"includeInShipping": null,
"isManual": null,
"createdDate": 1208983767866,
"createdByUserLogin": "admin",
"lastModifiedDate": null,
"lastModifiedByUserLogin": null,
"originalAdjustmentId": null,
"oldAmountPerQuantity": null,
"oldPercentage": null,
"_entityName": "OrderAdjustment"
}
],
"OrderContactMech": [
{
"orderId": "DEMO10090",
"contactMechPurposeTypeId": "BILLING_LOCATION",
"contactMechId": "9015",
"_entityName": "OrderContactMech"
},
{
"orderId": "DEMO10090",
"contactMechPurposeTypeId": "ORDER_EMAIL",
"contactMechId": "9026",
"_entityName": "OrderContactMech"
},
{
"orderId": "DEMO10090",
"contactMechPurposeTypeId": "SHIPPING_LOCATION",
"contactMechId": "9015",
"_entityName": "OrderContactMech"
}
],
"OrderItem": [
{
"orderId": "DEMO10090",
"orderItemSeqId": "00001",
"externalId": null,
"orderItemTypeId": "PRODUCT_ORDER_ITEM",
"orderItemGroupSeqId": null,
"isItemGroupPrimary": null,
"fromInventoryItemId": null,
"budgetId": null,
"budgetItemSeqId": null,
"productId": "CAM-2644",
"supplierProductId": null,
"productFeatureId": null,
"prodCatalogId": "DemoCatalog",
"productCategoryId": null,
"isPromo": "N",
"quoteId": null,
"quoteItemSeqId": null,
"shoppingListId": null,
"shoppingListItemSeqId": null,
"subscriptionId": null,
"deploymentId": null,
"quantity": 2.000000,
"cancelQuantity": null,
"selectedAmount": 0.000000,
"unitPrice": 79.990,
"unitListPrice": 89.990,
"unitAverageCost": null,
"unitRecurringPrice": null,
"isModifiedPrice": "N",
"recurringFreqUomId": null,
"itemDescription": "Nikon Analog Camera",
"comments": null,
"correspondingPoId": null,
"statusId": "ITEM_APPROVED",
"syncStatusId": null,
"estimatedShipDate": null,
"estimatedDeliveryDate": null,
"autoCancelDate": null,
"dontCancelSetDate": null,
"dontCancelSetUserLogin": null,
"shipBeforeDate": null,
"shipAfterDate": null,
"cancelBackOrderDate": null,
"overrideGlAccountId": null,
"salesOpportunityId": null,
"changeByUserLoginId": null,
"_entityName": "OrderItem",
"OrderAdjustment": [
{
"orderAdjustmentId": "9000",
"orderAdjustmentTypeId": "PROMOTION_ADJUSTMENT",
"orderId": "DEMO10090",
"orderItemSeqId": "00001",
"shipGroupSeqId": "_NA_",
"comments": null,
"description": null,
"amount": -38.400,
"recurringAmount": null,
"amountAlreadyIncluded": null,
"productPromoId": "9016",
"productPromoRuleId": "01",
"productPromoActionSeqId": "01",
"productFeatureId": null,
"correspondingProductId": null,
"taxAuthorityRateSeqId": null,
"sourceReferenceId": null,
"sourcePercentage": null,
"customerReferenceId": null,
"primaryGeoId": null,
"secondaryGeoId": null,
"exemptAmount": null,
"taxAuthGeoId": null,
"taxAuthPartyId": null,
"overrideGlAccountId": null,
"includeInTax": null,
"includeInShipping": null,
"isManual": null,
"createdDate": 1208983767866,
"createdByUserLogin": "admin",
"lastModifiedDate": null,
"lastModifiedByUserLogin": null,
"originalAdjustmentId": null,
"oldAmountPerQuantity": null,
"oldPercentage": null,
"_entityName": "OrderAdjustment"
}
],
"OrderItemPriceInfo": [
{
"orderItemPriceInfoId": "9000",
"orderId": "DEMO10090",
"orderItemSeqId": "00001",
"productPriceRuleId": "9000",
"productPriceActionSeqId": "01",
"modifyAmount": -9.600,
"description": "[PRODUCT_CATEGORY_IDIsPROMOTIONS] [list:89.99;avgCost:89.99;margin:0.0] [type:PRICE_POL]",
"rateCode": null,
"_entityName": "OrderItemPriceInfo"
}
],
"OrderItemShipGroupAssoc": [
{
"orderId": "DEMO10090",
"orderItemSeqId": "00001",
"shipGroupSeqId": "00001",
"quantity": 2.000000,
"cancelQuantity": null,
"_entityName": "OrderItemShipGroupAssoc"
}
],
"OrderItemShipGrpInvRes": [
{
"orderId": "DEMO10090",
"shipGroupSeqId": "00001",
"orderItemSeqId": "00001",
"inventoryItemId": "9001",
"reserveOrderEnumId": "INVRO_FIFO_REC",
"quantity": 2.000000,
"quantityNotAvailable": 0.000000,
"reservedDatetime": 1208983771474,
"createdDatetime": 1208983771474,
"promisedDatetime": 1210279767392,
"currentPromisedDate": null,
"priority": "2",
"sequenceId": null,
"oldPickStartDate": null,
"_entityName": "OrderItemShipGrpInvRes"
}
],
"OrderStatus": [
{
"orderStatusId": "9001",
"statusId": "ITEM_CREATED",
"orderId": "DEMO10090",
"orderItemSeqId": "00001",
"orderPaymentPreferenceId": null,
"statusDatetime": 1208983767392,
"statusUserLogin": "admin",
"changeReason": null,
"_entityName": "OrderStatus"
},
{
"orderStatusId": "9005",
"statusId": "ITEM_APPROVED",
"orderId": "DEMO10090",
"orderItemSeqId": "00001",
"orderPaymentPreferenceId": null,
"statusDatetime": 1208983773513,
"statusUserLogin": "admin",
"changeReason": null,
"_entityName": "OrderStatus"
}
]
}
],
// ...
}
entityExportReturn
Main record entity: ReturnHeader
Import API
Entity importers extend EntityImporter and implement the entityImportInterface service interface, are named entityImport* and are export=”true”:
<service name="entityImportInterface" engine="interface">
<description>Entity import from json-like record common interface for DataConnect services</description>
<attribute name="records" type="Object" mode="IN" optional="true">
<description>Record collection or entity list iterator</description>
</attribute>
<attribute name="maxOutRecords" type="Integer" mode="IN" optional="true">
<description>If specified, max entries for returned recordEntitiesMap</description>
</attribute>
<attribute name="recordEntitiesMap" type="Map" mode="OUT" optional="true">
<description>Map of short primary key values to lists of entities imported per record, one entry for each input record</description>
</attribute>
</service>
The exported (import) services require admin rights. Remotely with HTTPS, the standard service attributes “login.username”/”login.password” can be used to authenticate. An alternative set of non-exported services, entityImport*Internal without security checks are provided for situations where security is already provided.
Generally import services recognize roughly the same record definitions as their export equivalents. Some exceptions occur for importers such as the ProductInventory importer, whose main record isn’t defined by a (view-)entity. In Scipio’s entity importers system (not necessarily other places yet), view-entities can be imported (not just exported) as long as they are formed of simple joins, though complications may occur for complex records. Thanks to this, less custom coding is needed because the Data Connect API and Scipio now include ViewEntityUpdater logic which allows limited but automatic entity creation from view-entity definitions. This way, standard view-entity definitions can be reused and importers are easier to configure and write. Occasionally a view-entity must be enhanced with a member-entity-dependency xml element in the view-entity definition (see entitymodel.xsd) to resolve circular dependency detection or if the resolved order is incorrect or problematic.
Import services take a “records” IN attribute, typically a list of JSON-like records, where the record format for each importer is the same as the exporter format. In other words, it is usually a record represented by a main entity or view-entity’s fields to which are added custom fields and entity key records.
During import, keys in the main record that reflect entity names and used to automatically import any passed entities. The entities do not need to be related to the main record to be imported this way. In addition, import operations are supported for view-entities.
Only a limited number of custom keys exported by the exporters are recognized by the importers, so for the most part, importers automatically import the entities and view-entities from the main record and from keys named after entities, which are recursed. Client code may extend importers using entityprocessors.properties, but any new entity or view-entity can be imported without code modification by including the right entity keys in the JSON-like records. As such, listing the importers is mostly redundant.
Being export=”true” services they are published using WSDL/SOAP for remote communication (https://localhost:8443/admin/control/ServiceList?sel_service_name=entityImportCatalog&show_wsdl=true). For modern connections, the Data Connect webapp hosts HTTPS/JSON requests under the name: /dataconnect/control/entityImport*. This example creates a user having userLoginId “MyTestUser” and lets the system create an ID, which you’ll see in the returned data (Note: Do not use the default admin account outside of testing and a different account should be used on production):
curl https://localhost:8443/dataconnect/control/entityImportCustomer --insecure --header "Content-Type: application/json" --request POST --data '
{
"login.username" : "admin",
"login.password" : "scipio",
"records": [{"userLoginId" : "MyTestUser", "firstName" : "Great", "lastName" : "Example"}]
}'
{
"recordEntitiesMap": {
"10000": [
{
"partyTypeId": "PERSON",
"lastUpdatedStamp": 1602897024871,
"createdTxStamp": 1602897024865,
"createdStamp": 1602897024867,
"lastUpdatedTxStamp": 1602897024865,
"partyId": "10000"
},
{
"lastUpdatedStamp": 1602897024873,
"successiveFailedLogins": null,
"passwordHint": null,
"createdTxStamp": 1602894751648,
"createdStamp": 1602894751860,
"externalAuthId": null,
"lastUpdatedTxStamp": 1602897024865,
"lastTimeZone": null,
"enabled": null,
"lastLocale": null,
"currentPassword": null,
"userLoginId": "MyTestUser",
"isSystem": null,
"disabledDateTime": null,
"disabledBy": null,
"hasLoggedOut": null,
"userLdapDn": null,
"requirePasswordChange": null,
"lastCurrencyUom": null,
"partyId": "10000"
},
{
"firstName": "Great",
"lastName": "Example",
"lastUpdatedStamp": 1602897024874,
"createdTxStamp": 1602897024865,
"createdStamp": 1602897024874,
"lastUpdatedTxStamp": 1602897024865,
"partyId": "10000"
},
{
"lastUpdatedStamp": 1602897024877,
"roleTypeId": "CUSTOMER",
"createdTxStamp": 1602897024865,
"createdStamp": 1602897024877,
"lastUpdatedTxStamp": 1602897024865,
"partyId": "10000"
}
]
},
"_DEF_EVENT_MSG_": "Operation has been executed successfully",
"records": [
{
"userLoginId": "MyTestUser",
"firstName": "Great",
"lastName": "Example",
"partyTypeId": "PERSON",
"roleTypeId": "CUSTOMER"
}
],
"_EVENT_MESSAGE_": "Operation has been executed successfully"
}
The previous form lets the system create a partyId, here 10000. Afterward you can update fields on the same user by passing the ID you see in the returned records, on a stock setup:
curl https://localhost:8443/dataconnect/control/entityImportCustomer --insecure --header "Content-Type: application/json" --request POST --data '
{
"login.username" : "admin",
"login.password" : "scipio",
"records": [{"partyId": "10000", "lastName" : "Party"}]
}'