NAV Navbar
Logo
JSON cURL Python

Introduction

git clone git@github.com:Mathpix/api-examples.git
cd api-examples/images

The MathpixOCR API is a JSON API for extracting text from images. It is specialized to work on small to medium images (support for full pages coming soon), and has extensive support for scientific notation, as used in chemistry, math, physics, computer science, economics, and other STEM subjects.

To use MathpixOCR, you must base64 encode your image, and send it to the MathpixOCR API, along with other JSON information.

If you have any problems, please send us an email at support@mathpix.com.

Authorization

content-type: application/json
app_id: YOUR_APP_ID
app_key: YOUR_APP_KEY

MathpixOCR uses API keys to allow access to the API. You can find your API keys on your dashboard, which you can access by logging in or signing up.

MathpixOCR expects for the API key to be included in all API requests to the server via HTTP Basic Auth. Expected set of HTTP headers is shown on the right.

Processing a single image

Request options

You can request as many formats as you wish for a single image:

{
  "src":"data:image/jpeg;base64,...",
  "ocr": ["math", "text"], 
  "skip_recrop": true,
  "formats":[
    "text",
    "latex_simplified",
    "latex_styled",
    "mathml",
    "asciimath",
    "latex_list"
   ]
}
#!/usr/bin/env python
import sys
import base64
import requests
import json

# put desired file path here
file_path = 'limit.jpg'
image_uri = "data:image/jpg;base64," + base64.b64encode(open(file_path, "rb").read())
# (python3) image_uri = "data:image/jpg;base64," + base64.b64encode(open(file_path, "rb").read()).decode()
r = requests.post("https://api.mathpix.com/v3/latex",
    data=json.dumps({'src': image_uri}),
    headers={"app_id": "YOUR_APP_ID", "app_key": "YOUR_APP_KEY",
            "Content-type": "application/json"})
print json.dumps(json.loads(r.text), indent=4, sort_keys=True)
curl -X POST https://api.mathpix.com/v3/latex \
-H 'app_id: YOUR_APP_ID' \
-H 'app_key: YOUR_APP_KEY' \
-H 'Content-Type: application/json' \
--data '{ "src": "data:image/jpeg;base64,'$(base64 -i limit.jpg)'" }'

POST https://api.mathpix.com/v3/latex

Parameter Type Description
src string Image data, or public URL where image is located
ocr (optional) [string] The default is [“math”], to process all text pass [“math”, “text”]
formats [string] String postprocessing formats (see Formatting section).
skip_recrop (optional) bool Force algorithm to consider whole image
confidence_threshold (optional) number in [0,1] Set threshold for triggering confidence errors
region (optional)                                                                                                                                                                      object The region field, if present, specifies the image area to process using the properties top_left_x, top_left_y, width, and height, which are all in pixel coordinates (integer valued).
callback object Callback request object

Formatting options

The following formats can be used in the request:

Format Description
text to return LaTeX using text mode at the top level instead of math mode,
latex_normal                                                                                    for the most direct representation of the input
latex_simplified for modifications that assist symbolic processing such as shortening operator names, replacing long division with a fraction, and converting a column of operands into a single formula
latex_styled for modifications to improve the visual appearance such as adding ’\left’ and ’\right’ around parenthesized expressions that contain tall expressions like subscript or superscript;
latex_list to split output into a list of simplified strings that assist in processing multiple equations,
mathml to return a string containing the MathML for the recognized math; and
wolfram to return a string that is compatible with the Wolfram Alpha engine.

Result objects

{
    "detection_list": [],
    "detection_map": {
        "contains_chart": 0,
        "contains_diagram": 0,
        "contains_geometry": 0,
        "contains_graph": 0,
        "contains_table": 0,
        "is_inverted": 0,
        "is_not_math": 0,
        "is_printed": 0
    },
    "latex": "\\lim _ { x \\rightarrow 3 } ( \\frac { x ^ { 2 } + 9 } { x - 3 } )",
    "latex_confidence": 0.86757309488734,
    "position": {
        "height": 273,
        "top_left_x": 57,
        "top_left_y": 14,
        "width": 605
    }
}
Field Type Description
text (optional) string Predicted text format (not shown if not requested as format)
latex (optional) string Predicted latex format (not shown if not requested as format)
latex_list (optional) [string] Predicted latex_list format
wolfram (optional) [string] Predicted wolfram format
asciimath (optional) [string] Predicted asciimath format
position (optional) object Position object, pixel coordinates
detection_list (optional) [string] Detects image properties (see image properties)
error (optional) string US locale error message
error_info (optional) object Error info object
latex_confidence (optional) number in [0,1] Estimated probability 100% correct

Callback request object

Full request body example (includes callback field, shortened src field):

{  
   "src":"data:image/jpeg;base64,...",
   "ocr": ["math", "text"],
   "formats": ["text", "latex_styled"],
   "skip_recrop": true,
   "callback":{  
      "post":"http://RequestBin.com/10z3g561",
      "reply":"integral.jpg"
   }
}
Field Type Description
post string URL to which to make POST callback
headers object key value pairs of headers to make POST
reply (optional) string Sets values of reply field of callback response object (see callback response object)

Callback response object

{
  "reply": "integral.jpg",
  "result": {
    "detection_list": [
      "is_printed"
    ],
    "detection_map": {
      "contains_chart": 0,
      "contains_diagram": 0,
      "contains_geometry": 0,
      "contains_graph": 0,
      "contains_table": 0,
      "is_inverted": 0,
      "is_not_math": 0,
      "is_printed": 1
    },
    "error": "",
    "latex": "\\int \\frac { 4 x } { \\sqrt { x ^ { 2 } + 1 } } d x",
    "latex_confidence": 0.99817252453161,
    "position": {
      "height": 215,
      "top_left_x": 57,
      "top_left_y": 0,
      "width": 605
    }
  }
}
Field Type Description
reply string Request identifier
result object Result object

Image properties

The API defines multiple detection types:

Detection Definition
contains_diagram Contains a diagram.
is_printed The image is taken of printed math, not handwritten math.
is_not_math No valid equation was detected.

Error info object

In addition to the error field that contains a string (en-us locale) describing an error, a Mathpix response has an error_info field with an object providing programmatic information. The fields of this object include id, uniquely specifying the error, message, containing a string similar to the top-level error field, and detail fields specific to the type of error. The table below lists the different errors Mathpix returns.

Error info fields:

Field Type Description
id string specifies the error id (see below)
message string error message
detail (optional) string Additional error info

Error id types

Here is a table with all the error_id possibilities:

Id Description Detail fields HTTP Status
http_unauthorized                                            Invalid credentials 401
http_max_requests Too many requests count 429
json_syntax JSON syntax error 200
image_missing Missing URL in request body 200
image_download_error Error downloading image url 200
image_decode_error Cannot decode the image data 200
image_no_content No content found in image 200
image_not_supported Image is not math or text 200
image_max_size Image is too large to process 200
opts_bad_callback Bad callback field(s) post?, reply?, batch_id? 200
opts_unknown_ocr Unknown ocr option(s) ocr 200
opts_unknown_format Unknown format option(s) formats 200
math_confidence Low confidence 200
math_syntax Unrecognized math 200
batch_unknown_id Unknown batch id batch_id 200
sys_exception Server error batch_id? 200

Processing a batch of images

A batch request is made as follows:

{  
   "urls":{  
      "inverted":"https://raw.githubusercontent.com/Mathpix/api-examples/master/images/inverted.jpg",
      "algebra":"https://raw.githubusercontent.com/Mathpix/api-examples/master/images/algebra.jpg"
   },
   "callback":{  
      "post":"http://RequestBin.com/sr1x3lsr"
   }
}
curl -X POST https://api.mathpix.com/v3/batch \
-H "app_id: YOUR_APP_ID" \
-H "app_key: YOUR_APP_KEY" \
-H "Content-Type: application/json" \
--data '{ "urls": {"inverted": "https://raw.githubusercontent.com/Mathpix/api-examples/master/images/inverted.jpg", "algebra": "https://raw.githubusercontent.com/Mathpix/api-examples/master/images/algebra.jpg"},"callback":{"post": "http://RequestBin.com/sr1x3lsr"}}'

import requests
import json

base_url = 'https://raw.githubusercontent.com/Mathpix/api-examples/master/images/'

data = {
    'urls': {
        'algebra': base_url + 'algebra.jpg',
        'inverted': base_url + 'inverted.jpg'
    },
    'callback': {
        'post': 'http://RequestBin.com/1ftatkr1'
    }
}

r = requests.post(
    "https://api.mathpix.com/v3/batch", data=json.dumps(data),
    headers={
        'app_id': 'YOUR_APP_ID',
        'app_key': 'YOUR_APP_KEY',
        'content-type': 'application/json'
    },
    timeout=30
)
reply = json.loads(r.text)
assert reply.has_key('batch_id')

The callback response is as follows:

{
  "reply": {
    "batch_id": "11"
  }, 
  "result": {
    "algebra": {
      "detection_list": [], 
      "detection_map": {
        "contains_chart": 0, 
        "contains_diagram": 0, 
        "contains_geometry": 0, 
        "contains_graph": 0, 
        "contains_table": 0, 
        "is_inverted": 0, 
        "is_not_math": 0, 
        "is_printed": 0
      }, 
      "error": "", 
      "latex": "12 + 5 x - 8 = 12 x - 10", 
      "latex_confidence": 0.99640350138238, 
      "latex_list": [], 
      "position": {
        "height": 208, 
        "top_left_x": 0, 
        "top_left_y": 0, 
        "width": 1380
      }
    }, 
    "inverted": {
      "detection_list": [
        "is_inverted", 
        "is_printed"
      ], 
      "detection_map": {
        "contains_chart": 0, 
        "contains_diagram": 0, 
        "contains_geometry": 0, 
        "contains_graph": 0, 
        "contains_table": 0, 
        "is_inverted": 1, 
        "is_not_math": 0, 
        "is_printed": 1
      }, 
      "error": "", 
      "latex": "x ^ { 2 } + y ^ { 2 } = 9", 
      "latex_confidence": 0.99982263230866, 
      "latex_list": [], 
      "position": {
        "height": 170, 
        "top_left_x": 48, 
        "top_left_y": 85, 
        "width": 544
      }
    }
  }
}

The Mathpix API supports processing multiple images in a single POST request to a different endpoint: /v3/batch. The request contains urls specifying a key-url pair for each image, and callback that specifies where to send the results. The url in a key-url pair may either be a string or an object containing the url and additional image-specific request parameters such as region and formats. The response contains a unique batch_id value, and after processing all the images the server posts a message to callback.post containing result with a key-result pair for each image result and reply containing the batch_id.

An application may specify a value for callback.reply in the request to include fields in addition to batch_id in both the response and result post. However, the server-generated batch_id cannot be overridden.

Applications that wish to provide additional data in the post may specify a callback.body value. The server will pass this value to the post. If callback.headers is provided then the server will send the contents as headers in the post.

To check on the status of a batch request an application may use GET /v3/batch/:id where :id is the returned batch_id. Note that the GET request must contain the same app_id and app_key headers as the batch request. The response body is JSON containing the batch url keys (array of strings), current results (key-result objects), and the callback object passed to the original batch request. The server only keeps batch results for a limited amount of time after the batch is complete (currently two days but could change in the future).

Long division

{
    "detection_map": {
        "contains_chart": 0,
        "contains_diagram": 0,
        "contains_geometry": 0,
        "contains_graph": 0,
        "contains_table": 0,
        "is_inverted": 0,
        "is_not_math": 0,
        "is_printed": 1
    },
    "latex": "8 \\longdiv { 7200 }"
}

The following <script> tags will take care of rendering the \longdiv markup.

<script type="text/javascript" src="//cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/x-mathjax-config">
    MathJax.Hub.Config({
        tex2jax: {inlineMath: [['$','$'], ['\\(','\\)']]},
        messageStyle: "none",
        TeX: {
            Macros: {
               longdiv: ["{\\overline{\\smash{)}#1}}", 1]
            }
        }
    });
</script>

We use the special markup \longdiv to represent long division; it is the only nonvalid Latex markup we return. Long division is used much like \sqrt which is visually similar.

Latency considerations

The biggest source of latency is image uploads. The speed of a response from Mathpix API servers is roughly proportional to the size of the image. Try to use images under 100kb for maximum speeds. JPEG compression and image downsizing are recommended to ensure lowest possible latencies.

Example: Mathpix Snipping Tool

Mathpix Snipping Tool uses the JSON options seen on the right:

{
  "src":"data:image/jpeg;base64,...",
  "ocr": ["math", "text"], 
  "skip_recrop": true,
  "formats":[
      {
         "latex":[
            "styled",
            "rm_spaces"
         ]
      },
      {
         "text":[
            "text",
            "rm_spaces"
         ]
      }
   ]
}

Note: src image field is abbreviated to make the JSON more readable.

Vocabulary

Mathpix generates any of the following characters:

! $ &
( ) * + ,
- . / 0 1
2 3 4 5 6
7 8 9 : ;
< = > ? A
B C D E F
G H I J K
L M N O P
Q R S T U
V W X Y Z
[ \ ] ^ _
a b c d e
f g h i j
k l m n o
p q r s t
u v w x y
z { | } ~
\% \\ \{ \}
\alpha \angle \langle \rangle \approx
\because \begin{array} \beta \bot \cap
\cdot \cdots \chi \circ \cong
\cup \dagger \Delta \delta \div
\dot \dots \ell \emptyset \eta
\end{array} \epsilon \equiv \exists \kappa
\forall \frac \gamma \Gamma \geq
\hat \hbar \hline \in \infty
\int \Lambda \lambda \lceil \left(
\left. \left[ \left\{ \left| \Leftrightarrow
\leftrightarrow \lfloor \leq \longdiv \mathcal
\mp \mu \nabla \neq \notin
\oint \Omega \omega \operatorna \oplus
\overline \otimes \parallel \partial \perp
\Phi \phi \pi \pm \prime
\prod \propto \Psi \psi \qquad
\quad \rceil \rfloor \rho \right)
\right. \right\} \right] \Rightarrow \rightarrow
\right| \sigma \sim \simeq \sqrt
\square \star \sum \subset \supset
\subseteq \supseteq \tau \text \therefore
\Theta \theta \times \tilde
\varphi \vdots \vec \wedge \vee
\Xi \xi \zeta

Example outputs