12.1 Search

to start the search and

• Upload : Files uploaded through Data Gateway 

• Download : Files downloaded through Data Gateway 

• Push to Gateway : Files Transferred to Server.

• Push to Partner : Files Transferred from the Cloud    Bucket to remote Server.

• Pull from Partner : Files Transferred from the remote server to Cloud Bucket.

• Pull from Hub : Files Transferred from the server.

• File Arrived: Represents the time range during which the file was processed or transferred.
• Endpoint: Refers to the Partners-Id (Endpoint ID) associated with the file transfer.
• Bucket Name: The Cloud Storage Bucket or Container Name where the files are stored during the file transfer process.
• File Name: The name of the file that is involved in the transfer process.
• File Size: Displays the size of the transferred file in bytes, kilobytes, or megabytes.
• Source: Shows the file’s source location.
  • SFTP, FTP, FTPS, GCS, AWS, Azure etc.
• Status: Represents the outcome of the file transfer process, indicating whether the file transfer was Success or Failed.
• Flow: Describes the direction of the file transfer, such as:
  • Upload: Files uploaded through Data Gateway Web-UI.
  • Download: Files downloaded through Data Gateway Web-UI.
  • Push to Gateway: Files transferred to Server.
  • Push to Partner: Files transferred from the Cloud Bucket to remote Server.
  • Pull from Partner: Files transferred from the remote server to Cloud Bucket.
  • Pull from Hub: Files transferred from the server.
• Destination: Displays the destination location.
  • SFTP, FTP, FTPS, GCS, AWS, Azure etc.
• Account: Shows the Sender ID or Application User ID who initiated the file transfer.
• File Path: Displays the source file path from which the file was picked up for the transfer process.

12.2 Error codes and Debug steps

Error code /Status code	Debugging Steps
DG_0404	Double-check the file path or name. Ensure the file exists in the specified directory or location on your file system.
DG_1001	Ensure proper file reading loop termination; check for file existence before reading to end.
DG_1002	List directory to confirm file presence; validate file path using file explorer or SFTP client.
DG_1003	Review IAM or SSH user permissions; grant necessary permissions or elevate privileges if needed.
DG_1004	Check server logs for specific error details; add more logging around the failing operation for clarity.
DG_1005	Verify SFTP client and server versions; update outdated software components if required.
DG_1006	Test network connectivity with ping; restart networking services or troubleshoot network hardware.
DG_1007	Attempt to reconnect to server manually; monitor server logs for network-related errors or crashes.
DG_1008	Check SFTP server capabilities; find an alternative SFTP command or method for the unsupported operation.
DG_1009	Confirm handle creation before use; implement error handling for invalid handles.
DG_1010	Validate directory structure and file paths; create missing directories or correct file paths.
DG_1011	Check file listing to confirm duplicates; rename or move existing file or choose a new name for the upload.
DG_1012	Check file attributes using ls -l; change file permissions with chmod.
DG_1013	Ensure disk or storage availability; check server or system logs for disk-related errors.
DG_1014	Use df -h to check available space; clean up unnecessary files or expand storage.
DG_1015	Review user or group storage quotas; clear or archive files to free up space.
DG_1016	Verify user or group existence; add or correct user/group in permissions or ACL.
DG_1017	Identify locking processes using lsof; resolve or release conflicting locks.
DG_1018	List directory contents; backup and remove files or subdirectories.
DG_1019	Confirm file type with ls -l; check file path or change operation to file-centric.
DG_1020	Validate filename against naming rules; rename file to comply with allowed characters or length.
DG_1021	Trace symbolic links using read link; adjust or remove excessive links.
DG_1022	Check file attributes or permissions; remove read-only attributes or adjust permissions.
DG_1023	Review and validate parameters; refer to SFTP documentation or man pages for valid parameters.
DG_1024	Check file attributes or type; adjust operation to handle directories appropriately.
DG_1025	Identify conflicting locks with lsof; release or adjust locks causing conflicts.
DG_1026	Verify lock permissions or ownership; request appropriate permissions or adjust lock settings.
DG_1027	Check file status or attributes; complete or abort the pending delete operation.
DG_1028	Confirm corruption with file integrity tools; restore file from backup or re-fetch the file.
DG_1029	Validate owner’s existence and permissions; assign valid owner or adjust ownership settings.
DG_1030	Check group’s existence and membership; assign valid group or correct group membership.
DG_1031	Check servers and client logs for connection issues or errors. Temporarily disable firewalls/security software to rule out interference.
DG_1032	Check for network interruptions or timeouts; reconnect or restart the session; monitor for stability.
DG_1033	Check whether the credentials are correct or not and verify whether the user has enough permissions to access the server
DG_1034	Verify whether the SFTP is up and running using any third-party tools like WinSCP or FileZilla and if issue persists reach system admin.
DG_1035	This could be due to slow network response, server configuration issues, or authentication problems. To troubleshoot, check network stability, review server settings, and verify credentials for authentication.
DG_1036	Check for network stability, optimize server configurations, and ensure active communication between client and server to prevent idle connections from being closed prematurely.
DG_1037	Verify the SFTP server’s IP address or hostname and ensure it is correct. Check network connectivity and firewall settings to ensure there are no restrictions preventing the client from connecting to the server.
DG_1038	Ensure the SFTP server is running and listening on the correct port.  Verify whether the firewall allows traffic on SFTP port both inbound and outbound.
DG_2001	Check server status and network connectivity; ensure the server is running.
DG_2002	Switch between PASV and PORT modes; verify firewall or NAT settings.
DG_2003	Reconnect and resume transfer; review server logs for errors or timeouts.
DG_2004	Verify file status and path; ensure correct directory and file permissions.
DG_2005	Examine server logs; restart FTP session or server to clear errors.
DG_2006	Check server storage; clean up files or directories to free space.
DG_2007	Review FTP command syntax; check for typos or incorrect arguments.
DG_2008	Verify FTP server capabilities; use an alternative FTP command if needed.
DG_2009	Confirm correct FTP command sequence; execute commands in the proper order.
DG_2010	Check parameter values; refer to FTP documentation for supported parameters.
DG_2011	Verify username and password; reset password or contact server admin if needed.
DG_2012	Ensure the FTP server is running and listening to the correct port. Verify if the firewall allows traffic on FTP port (usually 21) both inbound and outbound.
DG_2013	Confirm file or directory existence; check permissions.
DG_2014	Review storage quotas; clean up or request more storage.
DG_2015	Rename file to comply with naming rules; remove forbidden characters.
DG_2016	Verify network stability; re-establish connection and consider resetting networking devices.
DG_2017	Switching to Passive Mode or updating network settings can resolve this issue.
DG_2018	Verify whether the FTP/FTPS is up and running using any third-party tools like WinSCP or FileZilla and if issue persists reach system admin.
DG_2019	Verify the hostname being used for the connection and ensure that the device has a working internet connection or try using a different DNS server.
DG_2020	Double-check the hostname for accuracy and verify the DNS configuration or network connectivity of the device.
DG_2021	Review server logs for potential issues and ensure network stability and server settings are optimized for seamless FTP connections.
DG_2022	Check credentials for correctness and permissions adequacy. Review firewall settings and network connectivity to resolve access denial.
DG_2023	Verify the hostname or IP address is correct and reachable. Check the DNS configuration or network connectivity to resolve the hostname.
DG_2024	Check whether the server details are valid or not, update the partner with valid details.
DG_3400	Check the request parameters and syntax. Ensure all required fields are provided and correctly formatted.
DG_3401	Verify authentication credentials (e.g., API keys, OAuth tokens). Ensure they are correctly included in the request headers or body.
DG_3403	Check permissions for the resource. Ensure the client has the necessary permissions to access it.
DG_3404	Verify the URL or resource path. Ensure it exists and is accessible.
DG_3405	Check the HTTP method used in the request against the allowed methods for the resource.
DG_3406	Review the Accept headers in the request. Ensure the server can produce a response in the requested format.
DG_3408	Check network connectivity. Ensure the client or server isn’t overloaded.
DG_3409	Identify the conflicting resource state. Modify the request to resolve the conflict.
DG_3410	Update or remove references to the deprecated resource.
DG_3413	Reduce request payload size. Check server configurations for request size limits.
DG_3415	Verify content type and format. Adjust request headers or body to match server expectations.
DG_3429	Implement retries with exponential backoff. Reduce request frequency or optimize batch requests.
DG_3451	Consult with legal teams. Ensure compliance with legal requirements.
DG_3500	Check server logs for details. Review server configurations and code for errors.
DG_3502	Check upstream server health. Review gateway or proxy configurations.
DG_3503	Check server health and resource utilization. Monitor for maintenance notices.
DG_3504	Review upstream server health. Adjust timeout settings if necessary.
DG_3507	Free up server storage. Optimize data storage and clean up old files.
DG_3511	Provide necessary network credentials. Check network configurations and authentication methods.
DG_4001	Verify whether the login attempt is made with valid credentials or not, contact admin for further assistance.
DG_4002	Contact admin to active the partner.
DG_4003	Contact admin to whitelist the IP.
DG_4004	Close existing protocol session, only open the number sessions which are allowed.
DG_4005	Contact admin to manage/edit partner.
DG_4006	Verify whether the login attempt is made with valid private key for SFTP protocol or not, contact admin for further assistance.
DG_4007	Try updating the password to ensure it is correctly encrypted with the appropriate algorithm and padding scheme, ensuring compatibility between encryption and decryption processes.
DG_5301	Verify and update the endpoint URL to match the bucket’s specified regional endpoint, like storage.googleapis.com/your-region-your-bucket
DG_5304	Check the If-Modified-Since or If-None-Match headers at your request to ensure they match the resource’s last modified timestamp or ETag. Confirm the resource hasn’t been modified since the specified condition; adjust the headers or request a fresh copy if needed.
DG_5307	Wait and retry accessing the bucket after some time; DNS propagation can take a while to complete.
DG_5308	Resume the upload from where it left off using the provided byte range to complete the upload successfully.
DG_5400	Increase the size of the upload to meet GCS’s minimum object size requirements before retrying the upload.
DG_5401	Provide valid authentication credentials (e.g., API key, OAuth token) in the request headers or body and retry the request.
DG_5403	Check permissions and ensure you have the necessary access rights for the object. Adjust permissions or request access from the owner.
DG_5404	Verify the bucket name spelling and existence in your GCS account. Create the bucket if it doesn’t exist or correct the bucket name in your request.
DG_5405	Review GCS documentation to determine supported methods for the resource. Adjust the request to use an allowed HTTP method or ensure correct bucket and object names.
DG_5408	Implement exponential backoff in the retry mechanism when uploading to GCS. This helps handle transient network issues gracefully.
DG_5409	select a different name and try again. This error can occur when you try to create a bucket name that already exists, or you try to create a bucket name that does not meet the bucket naming requirements.
DG_5411	Add the Content-Length header to your request with the correct size of the content being sent. Ensure it accurately reflects the size of the payload.
DG_5412	Review the pre-conditions specified in the request, such as If-Match, If-None-Match, If-Modified-Since, or If-Unmodified-Since. Adjust the conditions or ensure they match the resource’s current state for the request to succeed.
DG_5413	Optimize the copy operation or increase the request timeout. Consider using gsutil with -m option for parallel copying or chunking the large object to speed up the process.
DG_5416	Check the byte range specified in the request. Ensure it’s within the valid range of the resource. Adjust the range or request the full resource if necessary.
DG_5429	Implement exponential backoff in the retry mechanism when interacting with Cloud Storage to handle rate limit exceeded errors gracefully.
DG_5500	Retry the request using exponential backoff. If the issue persists, check the GCS status dashboard or logs for any ongoing issues or outages.
DG_5504	Retry the request using exponential backoff. Check network connectivity and service availability. If the issue persists, verify the service’s health and consider checking logs for any related errors.
DG_5505	Verify billing account status in Google Cloud Console and ensure it’s closed. Reactivate the closed billing account by resolving any outstanding payment issues or administrative actions through Google Cloud Console.
DG_6301	Use the bucket’s specific regional endpoint provided by AWS and update your AWS SDK or client configuration to match the endpoint.
DG_6304	Verify the request headers for If-Modified-Since or If-None-Match to ensure they match the resource’s current state and retry the request if needed.
DG_6307	Wait and retry accessing the bucket after DNS propagation is complete or use the bucket’s specific regional endpoint if available.
DG_6400	Increase the upload size to meet AWS S3’s minimum requirement or adjust the upload method accordingly.
DG_6401	Check IAM permissions or S3 bucket policies. Ensure the user or role has the necessary permissions for the operation.
DG_6403	Choose a supported region for S3 Storage Lens or verify the AWS documentation for the current list of supported regions.
DG_6404	Verify the bucket name spelling and existence in your AWS account. Ensure you’re using the correct AWS region.
DG_6405	Check AWS S3 documentation to confirm supported methods for the resource. Adjust the request to use an allowed HTTP method.
DG_6409	Choose a unique bucket name that hasn’t been taken by any AWS S3 user.
DG_6411	Add the Content-Length header to your request with the correct size of the content being sent.
DG_6412	Review and adjust the preconditions specified in the request, such as If-Match, If-None-Match, If-Modified-Since, or If-Unmodified-Since, to match the resource’s current state.
DG_6416	Adjust the byte range specified in the request to ensure it falls within the valid range for the resource.
DG_6503	Implement an exponential backoff in your request retries or reduce the frequency of requests to stay within AWS S3’s rate limits.
DG_6500	Retry the operation after a short delay. If the issue persists, check AWS service health dashboard or contact AWS support for further assistance.
DG_6501	Remove or replace the unsupported header with a valid one according to AWS S3 documentation.
DG_6507	Check the available storage capacity in your AWS S3 account. Consider deleting unused objects or upgrading your storage plan.
DG_7307	Verify the current endpoint you are using for accessing the blob or container. Update your application or client to follow the temporary redirect to the new endpoint provided in the error message.
DG_7308	Verify the endpoint you are using for accessing the blob or container. Update your application or client to use the new permanent endpoint provided in the error message for all future requests.
DG_7400	Validate the data being uploaded against Azure Blob Storage requirements. Retry the upload operation with corrected or valid content.
DG_7401	Check IAM permissions or azure policies. Ensure the user or role has the necessary permissions for the operation.
DG_7403	Verify the permissions for the user or service principal making the request. Adjust permissions to allow blob overwrites or use a different blob name for
DG_7404	Verify the blob name or ID provided in the request. Ensure the blob is in the correct container or check if it was deleted or
DG_7409	Use a different name or ID for the blob to avoid collision. Double-check the container to ensure the blob was not accidentally created.
DG_7412	Ensure the specified position meets the condition set for the append. Adjust the position or conditions and try the append operation again
DG_7416	Verify the page range provided in the request. Modify the range to fit within the blob’s size and try the operation again.
DG_7500	Verify the source blob/container details provided for the copy operation. Attempt the copy operation again, ensuring the source is accessible and valid.
DG_8307	Wait for a few minutes and try accessing the bucket again. Ensure that your DNS settings are correctly configured to resolve the Object Storage endpoint.
DG_8400	Review the minimum object size requirement for the storage class you’re using in IBM Cloud Object Storage. Ensure the object you’re uploading meets or exceeds the minimum size specified for the storage class.
DG_8403	Verify that you’ve entered the correct access key ID. Ensure that the access key ID is still active and hasn’t been deleted or deactivated
DG_8404	Double-check the bucket name you provided. Verify that the bucket hasn’t been deleted or renamed.
DG_8405	Review IBM’s documentation to determine which HTTP methods are supported for the resource. Check your request method and adjust it accordingly.
DG_8409	Choose a different bucket name that’s unique. Consider adding a prefix or suffix to the bucket name to make it unique.
DG_8500	Try the operation again after waiting for a few minutes. If the error persists, contact IBM with the error details. Check for application logs.
DG_8501	Review the request headers and ensure you’re not using unsupported features. Check IBM’s documentation for supported headers and adjust your request accordingly.
DG_8503	Reduce the rate of your requests.  Implement an exponential backoff for retries in your application. Check IBM’s status page for any ongoing outages or maintenance.
DG_8601	Check network connection and speed, if issue persist reach out to admin

12.3 Upload

The Upload page is used to upload files directly into the Google Cloud Storage. This menu is available only or File Manager and File Operator roles. The available buckets for upload are configured during user creation in the application. User can find the list of folders which he/she has access to upload. You can select the folder to which the file needs to be uploaded and in the right pane, you can click to browse files from your local or drag and drop the file to upload the file. Once done, click on upload icon for each of the file or hit the UPLOAD ALL button. The files will be uploaded and stored into the destination Cloud Bucket. Also, the transaction is tracked, and you can view it in the Search module.

12.4 Download

The Download page is used to download files directly from the Google Cloud Storage. This menu is available only for File Manager and File Operator roles. The available buckets for download are configured during user creation in the application. User can find the list of folders which he/she has access to download. You can select the folder from which the file needs to be downloaded and in the right pane, you will be able to see all the available files with the Download link as well. You can click on the download link and download the required files. Transaction details, including successful downloads, are recorded and can be viewed in the Transaction Search page.

12.5 File Replay

The File Replay feature within the Data Gateway application is a powerful tool designed to restore deleted files to their original folders and reprocess them to their intended destinations. This functionality is seamlessly integrated into the Transaction Search page, simplifying the restoration and reprocessing process. The File Replay feature is only applicable to the Pull from Hub and Push to Partner transactions. In these two scenarios, source files are available within the server and archived for the period specified in the Endpoint. During this period, you will be able to do file replay which will fetch the file from archive and does the reprocess. Below are the steps for performing a File Replay 1. Navigate to the Transaction Search page. 2. Locate the desired files under Pull from Hub or Push to Partner scenarios. 3. Select the files that need to be reprocessed. 4. Click on the Reprocess button. 5. The selected files will be eligible for reprocessing, and their status will be updated to Reprocessed. 6. The files will be restored and processed again to their original destination paths. In case, you select files which are of other directions (Pull from Partner, Push to Gateway), you will be getting notifications mentioning those are not eligible for reprocess.