Our Thinking

This is a forum to share our thoughts and get your feedback.

  • Home
    Home This is where you can find all the blog posts throughout the site.
  • Categories
    Categories Displays a list of categories from this blog.
  • Tags
    Tags Displays a list of tags that have been used in the blog.
  • Bloggers
    Bloggers Search for your favorite blogger from this site.
  • Team Blogs
    Team Blogs Find your favorite team blogs here.
  • Login
    Login Login form

Bulk Uploads to CRMOD

Posted by on in Oracle CRM On Demand
  • Font size: Larger Smaller
  • Hits: 1444
  • 0 Comments
  • Subscribe to this entry
  • Print

Often client applications need to insert/update millions of records such as leads, orders etc into CRMOD. CRMOD regular web services allow max 20 records per request, so if you want to push 1M activities, you have to make 50,000 calls, which can be painful!

As an example, consider StoryPulse, our flagship product that supports all popular CRM platforms. The platform collects millions of assessments from thousands of CMROD users from their mobile devices. Once the assessments are in the middle tier, the platform pushes them to respective backend; CRMOD in this case.

However, there are few problems with regular web services. If we were to use them:

  1. It takes longer time to push tasks to CRMOD.
  2. Thousands of CRMOD calls need to be made.
  3. There could be be issues with session
    1. max sessions reached
    2. max 20 web service calls in a given timeframe etc

 

So, we thought of using CRMOD batch processing to push millions of records in a faster and better way.

 

CRMOD has several utilities to import bulk data; online import assistant, data loader, migration tool etc. Data loader is a client utility that accepts huge data sets in a CSV file and inserts them into the CRMOD environment.

 

Client applications can also take advantage of this API and invoke web service calls to push bulk data. While customers are moving away into Oracle Sales Cloud, Salesforce etc, I thought I will take a moment to explain the setup of the this API and push data to CRMOD.

 

Bulk API Setup and access

 

Any admin user can download data loader from their CMROD UI. As part the utility a WSDL file “OracleDataLoaderOnDemandImportServices.wsdl” gets download. All you need to do is use this WSDL and setup your application to insert bulk data. It has following methods.

 

  1. BulkOpImportCreateRequest
    • It creates a basic bulk import request object.
    • In this object we have to mention the following:
      1. Name of the object such as “Account” or “Task”
      2. What operation you want to perform (like insert or update).
      3. CSV file name which as all the data.
    • This method needs to be called prior you make one or more send data calls.
  2. BulkOpImportSendData
    • This method will push the data to the server.
    • It will contain actual data sets, which needs to be pushed.
  3. BulkOpImportGetRequestDetail
    • Using this method, we can get a request status, how many records are failed to process, error messages etc

 

Please refer following Data Loading API URLs for more information on each of these methods.

 

Using this bulk import we can push 30,000 records (Eg – Assessment Activities ) per request. This number may be vary for different record and industry. If we can run this process using multiple threads and a proper session pool, we can insert millions of records in very few requests and less time.

 

Limitations -

 

  1. Imported file should be CSV only.
  2. If any of the records in the request cause an error, we can’t revert the records inserted before.
  3. Only one type of record can be inserted at a time.
  4. After batch job completes client applications does not have records Row-id details, need to query them using other call with External Unique Id reference.

 

In this blog, I could explore and explain bulk insertion with respect to our problem in the StoryPulse application. Your use case could be different but you can refer to the links below.

 

https://docs.oracle.com/cd/E67776_01/books/OnDemWebSvcs/OnDemWebSvcs_SiebelAPICalls134.html

 

http://docs.oracle.com/cd/E35875_01/books/OnDemOLH/index.htm?toc.htm?importingdatahelp.html

 

 

 

0

Comments

  • No comments made yet. Be the first to submit a comment

Leave your comment

Guest
Guest Friday, 14 December 2018

Archive

January
February
March
May
July
August
September
October
November
December
January
February
March
June
August
September
November
December
January
March
April
May
June
August
October
November
December
January
February
March
April
May
June
July
November
December
Go to top