CloudWorks Integration with BigQuery: Understanding the Error Message Table

Hi Team,

In the official documentation under "Google BigQuery Permissions"

I found details about creating an optional error message table with the following structure:

  • integration_id
  • run_id
  • run_date
  • error_code
  • error_message
  • failed_row

While the documentation outlines the table structure, it lacks details on how records are populated. Will this table capture both import and export error logs, similar to an audit entry?

I attempted to trigger errors in both import and export integrations within CloudWorks to test if records would be inserted into this table. However, no inserts occurred. Also the actual import and export integrations function worked properly, so there are no permission issues with the service account.

Does anyone have insights into the purpose of this error message table and how it functions?

Thanks

Answers

  • The error message table in CloudWorks is designed to record integration issues for troubleshooting and improvement. It may not Spotify Receiptify capture all errors. To investigate, review CloudWorks documentation, check integration logs, and contact support if needed. Analyzing the table can help identify recurring issues and improve data integration processes.