Bulk Export of Opportunity Data from Salesforce to AWS

In an era of integrated platforms, businesses often rely on multiple systems to cater to different functions. For instance, while Salesforce is a powerful tool for sales and CRM, enterprises might use different ERP systems to manage other aspects of their operations. In our case, a global manufacturing enterprise, sought to integrate its Salesforce and ERP systems, using AWS as the intermediate platform.

Problem Statement:

Manufacturing business sales teams use Salesforce to manage their leads, opportunities, and deals. Once an opportunity is closed in Salesforce, the details need to be pushed into the company’s ERP system (SAP) for further processing, like invoicing and delivery management. Transferring this data manually is inefficient and error-prone.

Solution:

Bulk API Export in Salesforce:

• Salesforce offers Bulk API, a tool designed to make it simple to process data from Salesforce.
• A scheduled job was set up within Salesforce to use the Bulk API for exporting closed opportunity data as CSV files on a daily basis.

Amazon S3 as Storage:

• AWS’s Amazon S3 was chosen as the destination for storing these exported CSV files due to its durability, scalability, and easy integration capabilities.
• The exported CSV files from Salesforce were directly uploaded to a dedicated S3 bucket.

AWS Lambda for Processing:

• An AWS Lambda function was set up to trigger whenever a new CSV file was uploaded to the S3 bucket.
• This Lambda function processed the CSV file and pushed the data to the downstream ERP system using its API.

Implementation Details:

• Salesforce’s scheduled job, once executed, initiates the Bulk API request to extract closed opportunity data, which then gets saved as a CSV file.
• An AWS SDK within Salesforce was then used to upload this CSV to a predefined S3 bucket.
• S3 Event Notification was configured to invoke the AWS Lambda function upon the arrival of a new CSV.
• The Lambda function parsed the CSV file and sent the data to the ERP system.

Security & Data Integrity:

• Secure transfer mechanisms (like S3 Transfer Acceleration) were used to ensure data security while uploading.
• AWS Identity and Access Management (IAM) ensured that only authorized personnel and services could access the S3 bucket.
• Data validation checks were performed both during the extraction from Salesforce and before pushing data to the ERP.

Monitoring & Alerts:

• Amazon CloudWatch was set up to monitor the entire process, tracking any failures in data upload or processing.
• Alerts were configured to notify the system administrators in case of any issues, ensuring prompt resolutions.

Results:

• Automation: Manual handoff between Salesforce and the ERP system was completely eliminated.
• Efficiency: Data transfer was streamlined, reducing the time lag between a closed opportunity in Salesforce and its processing in the ERP.
• Reliability: With AWS services handling the integration, data loss was minimized, and any issues were quickly addressed.

Conclusion:

By leveraging the capabilities of Salesforce’s Bulk API and the extensive suite of AWS services, the Manufacturing company was able to establish a seamless and efficient integration between its CRM and ERP systems. This not only eliminated manual work but also ensured timely and reliable data transfers.