top of page
Search

Extract Text from Image – Einstein OCR

Writer: goldenstateservicesj
goldenstateservicesj
Jun 2, 2020
3 min read

Hi there! 🙋‍♂️

As promised in my introductory article on Einstein OCR, today we are going to discuss how we can simply extract the text from an image using Einstein OCR. Before we dwell any further, I will just briefly discuss how the Vision API works.

So the first thing that you need is an account. To get yourself started, click here. Once you have an account, we can move forward to address a somwhat complicated question and i.e. how do we authenticate?

FYI, Einstein Platform Services APIs(both Vision and Language) use OAuth 2.0 JWT bearer token flow for authorization.

There are two ways to authenticate:

  • Use the token page to upload your key file and generate a JWT token.

  • Write some Apex to setup the authentication.

I chose the latter because if you choose the first, you will have generate the token everytime it gets expired from the token page. Additionally, I was curious to understand how we can setup OAuth 2.0 JWT bearer token flow using Apex, hence the latter.

Read more on authentication, here.

Once we have the token we can start with defining our callout and in order to do that we will need to refer the API reference doc for Einstein OCR endpoint specifications.

The endpoing basically accepts a modelId along with image data. You can pass this image data as:

  • Binary content of image file uploaded as multipart/form-data, or

  • URL of the publically hosted image, or

  • Base64 encoded image data.

Now we’re all set to define our callout and it is going to look something like this:

/* In this method, I'm using a utility class named 'HttpFormBuilder' * to build the HTTP request body in multipart/form-data format. */public static String detectText(String sample, String detectionType, String accessToken, String model, boolean isBase64) { String OCREndpoint = 'https://api.einstein.ai/v2/vision/ocr'; string contentType = HttpFormBuilder.GetContentType(); // Compose the form string form64 = ''; form64 += HttpFormBuilder.WriteBoundary(); form64 += HttpFormBuilder.WriteBodyParameter('modelId', EncodingUtil.urlEncode(model, 'UTF-8')); form64 += HttpFormBuilder.WriteBoundary(); form64 += HttpFormBuilder.WriteBodyParameter('task', EncodingUtil.urlEncode(detectionType, 'UTF-8')); form64 += HttpFormBuilder.WriteBoundary(); if(isBase64) { form64 += HttpFormBuilder.WriteBodyParameter('sampleBase64Content', sample); } else { form64 += HttpFormBuilder.WriteBodyParameter('sampleLocation', sample); } form64 += HttpFormBuilder.WriteBoundary(HttpFormBuilder.EndingType.CrLf); blob formBlob = EncodingUtil.base64Decode(form64); string contentLength = string.valueOf(formBlob.size()); HttpRequest httpRequest = new HttpRequest(); httpRequest.setBodyAsBlob(formBlob); httpRequest.setHeader('Connection', 'keep-alive'); httpRequest.setHeader('Content-Length', contentLength); httpRequest.setHeader('Content-Type', contentType); httpRequest.setMethod('POST'); httpRequest.setHeader('Authorization','Bearer ' + accessToken); httpRequest.setEndpoint(OCREndpoint); Http http = new Http(); HTTPResponse res = http.send(httpRequest); System.debug(res.getStatus()); String responseBody; if (res.getStatusCode() == 200) { responseBody = res.getBody(); } return responseBody; }

API Response

Let’s see what will be the expected JSON response that we can expect to receive from the callout made.

{ "task": "text", "probabilities": [ { "probability": 0.99937266, "label": "Word1", "boundingBox": { "minX": 582, "minY": 685, "maxX": 1151, "maxY": 815 } }, { "probability": 0.99937266, "label": "Word2", "boundingBox": { "minX": 582, "minY": 685, "maxX": 1151, "maxY": 815 } } ], "object": "predictresponse"}

So from above dummy response, you will notice that the ‘probabilities’ property, which is of list type, will contain all the extracted words.

Let’s see some action

Some more cool and exciting use cases for Einstein OCR coming up in my future postsso stay tuned! 😎

Hope you find this one useful! See you in the next one! ✌ Thank you for being an awesome reader! Subscribe to this blog for receiving all the latest updates straight to your inbox. 🙂

 
 
 

Comments

bottom of page