Skip links

Einstein Vision: How to Build AI-Powered Apps, Fast

Jump To Section

Have you ever wondered how to use Salesforce Einstein to derive meaningful insights from your unstructured data, such as images and text data? Explore this example to learn how to integrate Einstein Platform Services into your CRM workflows to build AI-powered apps, fast.

Have you ever wondered how to use Salesforce Einstein to derive meaningful insights from your unstructured data, such as images and text data? Explore this example to learn how to integrate Einstein Platform Services into your CRM workflows to build AI-powered apps, fast.

We at Altimetrik have built a hypothetical app called, Who am I?, using Einstein Platform Services. Who am I? uses Einstein Vision’s image recognition technology, specifically Einstein Image Classification, to recognize and discern content based on images across several industries within the Fortune 500. Let’s get started to see how it works on a simple example!

Gather Training Data

To get started with Einstein Image Classification, you’ll first need to create a training dataset with defined labels. Each folder will contain sample images that correspond to your training labels. Once you’ve gathered your training data, you can save then save it as a .zip file and  upload it to Dropbox before making an API call to the Einstein Vision API (cURL command).


Folder structure of a sample dataset
curl -X POST -H "Authorization: Bearer " -H "Cache-Control: no-cache" -H "Content-Type: multipart/form-data" -F "type=image" -F "path=https://www.dropbox.com/s/abcdxyz/ImageRecognizerDataSet.zip?dl=1" https://api.einstein.ai/v2/vision/datasets/upload/sync

The above command will return the newly created Dataset ID and Label IDs along with the Label Name and number of examples that correspond to each training label. The type parameter in the cURL command determines the model type and are as follows:

image
image-detection (available in Einstein Vision API version 2.0 and later)
image-multi-label (available in Einstein Vision API version 2.0 and later)

Train your Model

As mentioned, training your dataset requires making a simple API call. You can use the cURL command below (where “<DATASET_ID>” is the ID returned from the dataset creation):

curl -X POST -H "Authorization: Bearer " -H "Cache-Control: no-cache" -H "Content-Type: multipart/form-data" -F "name=Image Recognizer Model" -F "datasetId=" https://api.einstein.ai/v2/vision/train

Sample response of training status

Start Classifying Images

You can use the command below with the Model ID and image location to classify new images.

curl -X POST -H “Authorization: Bearer ” -H “Cache-Control: no-cache” -H “Content-Type: multipart/form-data” -F
“sampleLocation=
-F “modelId=”

This command returns a prediction i.e. the probability that the new image falls within one of your defined labels.

And, because Who am I? is a mobile app, it’s optimized to make predictions from mobile images. The following is a sample callout for using data from mobile devices, as we know many customers, like us, prefer to build mobile first apps.

curl -X POST -H “Authorization: Bearer ” -H “Cache-Control: no-cache” -H “Content-Type: multipart/form-data” -F “sampleBase64Content=/9j/4AAQSkZ…” -F “modelId=YCQ4ZACEPJFGXZNRA6ERF3GL5E” https://api.einstein.ai/v2/vision/predict

Building AI-Powered Apps

Now that you’ve trained your model, it’s time to see Einstein Image Classification in action.

Using your mobile phone, capture any image such as a picture of a leaf in this example:

Einstein Image Classification will then return the prediction with a probability range as seen in the image.
Looks like our app identified a mulberry leaf with 90% confidence!

After getting a basic app working, the next step is to embed Einstein Image Classification into your application workflows to create AI-powered apps for your business. If you want to learn more about how to integrate intelligence into your applications, make sure to attend the following sessions:
Test Automation and Continuous Integration in SDLC Using Salesforce DX, by Oscar Quintela, 1:00 PM to 1:40 PM at Moscone West 2009
Codeless Advanced Lightning Components, by Fernanda Vecino and Rodrigo Juliani, 3:00 PM to 3:40 PM at Moscone West 2009
Smart Lookups with Machine Learning, by Juan Bessonart and Diego Latierro, 2:15 PM to 2:35 PM at Moscone West, Developer Theater

About Author

Chitra is an expert in development and deployment of cloud computing technologies and has managed global teams. She has led the deployment of cloud computing platforms for Fortune 100 companies in varied sectors – Pharmaceutical, Insurance, Automobile, Aviation, Telecom, to state a few.

She has experience in integrating IoT and cloud platform in the renewable energy sector. Chitra holds an M.S. in Computer Science from Ohio University, USA and B. Tech in Information Technology from the University of Madras, India.

admin

admin

Latest Reads

Subscribe

Suggested Reading

Ready to Unlock Yours Enterprise's Full Potential?

Adaptive Clinical Trial Designs: Modify trials based on interim results for faster identification of effective drugs.Identify effective drugs faster with data analytics and machine learning algorithms to analyze interim trial results and modify.
Real-World Evidence (RWE) Integration: Supplement trial data with real-world insights for drug effectiveness and safety.Supplement trial data with real-world insights for drug effectiveness and safety.
Biomarker Identification and Validation: Validate biomarkers predicting treatment response for targeted therapies.Utilize bioinformatics and computational biology to validate biomarkers predicting treatment response for targeted therapies.
Collaborative Clinical Research Networks: Establish networks for better patient recruitment and data sharing.Leverage cloud-based platforms and collaborative software to establish networks for better patient recruitment and data sharing.
Master Protocols and Basket Trials: Evaluate multiple drugs in one trial for efficient drug development.Implement electronic data capture systems and digital platforms to efficiently manage and evaluate multiple drugs or drug combinations within a single trial, enabling more streamlined drug development
Remote and Decentralized Trials: Embrace virtual trials for broader patient participation.Embrace telemedicine, virtual monitoring, and digital health tools to conduct remote and decentralized trials, allowing patients to participate from home and reducing the need for frequent in-person visits
Patient-Centric Trials: Design trials with patient needs in mind for better recruitment and retention.Develop patient-centric mobile apps and web portals that provide trial information, virtual support groups, and patient-reported outcome tracking to enhance patient engagement, recruitment, and retention
Regulatory Engagement and Expedited Review Pathways: Engage regulators early for faster approvals.Utilize digital communication tools to engage regulatory agencies early in the drug development process, enabling faster feedback and exploration of expedited review pathways for accelerated approvals
Companion Diagnostics Development: Develop diagnostics for targeted recruitment and personalized treatment.Implement bioinformatics and genomics technologies to develop companion diagnostics that can identify patient subpopulations likely to benefit from the drug, aiding in targeted recruitment and personalized treatment
Data Standardization and Interoperability: Ensure seamless data exchange among research sites.Utilize interoperable electronic health record systems and health data standards to ensure seamless data exchange among different research sites, promoting efficient data aggregation and analysis
Use of AI and Predictive Analytics: Apply AI for drug candidate identification and data analysis.Leverage AI algorithms and predictive analytics to analyze large datasets, identify potential drug candidates, optimize trial designs, and predict treatment outcomes, accelerating the drug development process
R&D Investments: Improve the drug or expand indicationsUtilize computational modelling and simulation techniques to accelerate drug discovery and optimize drug development processes