NAV Navbar
Logo
JSON cURL Python

Introduction

git clone git@github.com:Mathpix/api-examples.git
cd api-examples/images

The MathpixOCR API is a JSON API for extracting text from images. It is specialized to work on small to medium images (support for full pages coming soon), and has extensive support for scientific notation, as used in chemistry, math, physics, computer science, economics, and other STEM subjects.

To use MathpixOCR, you must base64 encode your image, and send it to the MathpixOCR API, along with other JSON information.

If you have any problems, please send us an email at support@mathpix.com.

Authorization

The request headers must be set as follows:

{
  "content-type": "application/json",
  "app_id": "YOUR_APP_ID",
  "app_key": "YOUR_APP_KEY"
}

MathpixOCR uses API keys to allow access to the API. You can find your API keys on our dashboard website by signing up at https://dashboard.mathpix.com.

MathpixOCR expects for the API key to be included in all API requests to the server via HTTP Basic Auth. Expected set of HTTP headers is shown on the right.

Process image (v3/latex)

Mathpix supports image recognition for jpg and png images.

Request parameters

You can request multiple formats for a single image:

{
  "src": "data:image/jpeg;base64,...",
  "ocr": ["math", "text"], 
  "skip_recrop": true,
  "formats": [
    "text",
    "latex_simplified",
    "latex_styled",
    "mathml",
    "asciimath",
    "latex_list"
   ]
}
#!/usr/bin/env python
import sys
import base64
import requests
import json

# put desired file path here
file_path = 'limit.jpg'
image_uri = "data:image/jpg;base64," + base64.b64encode(open(file_path, "rb").read())
# (python3) image_uri = "data:image/jpg;base64," + base64.b64encode(open(file_path, "rb").read()).decode()
r = requests.post("https://api.mathpix.com/v3/latex",
    data=json.dumps({'src': image_uri, 'formats': ['latex_normal']}),
    headers={"app_id": "YOUR_APP_ID", "app_key": "YOUR_APP_KEY",
             "Content-type": "application/json"})
print json.dumps(json.loads(r.text), indent=4, sort_keys=True)
curl -X POST https://api.mathpix.com/v3/latex \
-H 'app_id: YOUR_APP_ID' \
-H 'app_key: YOUR_APP_KEY' \
-H 'Content-Type: application/json' \
--data '{ "src": "data:image/jpeg;base64,'$(base64 -i limit.jpg)'", "formats": ["latex_normal"] }'

POST https://api.mathpix.com/v3/latex

Parameter Type Description
src string Image data, or public URL where image is located
formats string[] String postprocessing formats (see Formatting section)
ocr (optional) string[] Process only math [“math”] or both math and text [“math”, “text”]
format_options (optional) object Options for specific formats (see Formatting section)
skip_recrop (optional) bool Force algorithm to consider whole image
confidence_threshold (optional) number in [0,1] Set threshold for triggering confidence errors
beam_size (optional) number in [1,5] Number of results to consider during recognition
n_best (optional) number in [1,beam_size] Number of highest-confidence results to return
region (optional) object Specify the image area with the pixel coordinates top_left_x, top_left_y, width, and height
callback (optional) object Callback request object

Formatting

The following formats can be used in the request:

Format Description
text text mode output, with math inside delimiters, eg. test \(x^2\), inline math by default
text_display same as text, except uses block mode math instead of inline mode when in doubt
latex_normal direct LaTeX representation of the input
latex_styled modified output to improve the visual appearance such as adding ’\left’ and ’\right’ around parenthesized expressions that contain tall expressions like subscript or superscript
latex_simplified modified output for symbolic processing such as shortening operator names, replacing long division with a fraction, and converting a column of operands into a single formula
latex_list output split into a list of simplified strings to help process multiple equations
mathml the MathML for the recognized math
asciimath the AsciiMath for the recognized math
wolfram a string compatible with the Wolfram Alpha engine

Format options

To return a more compact latex_styled result, one could send the following request:

{
  "src":"data:image/jpeg;base64,...",
  "ocr": ["math", "text"], 
  "skip_recrop": true,
  "formats": [
    "text",
    "latex_simplified",
    "latex_styled",
    "mathml",
    "asciimath",
    "latex_list"
   ],
   "format_options": {
     "latex_styled": {"transforms": ["rm_spaces"]}
   }
}

The result for “latex_styled” would now be
    "\\lim_{x \\rightarrow 3}\\left(\\frac{x^{2}+9}{x-3}\\right)"
instead of
    "\\lim _ { x \\rightarrow 3 } \\left( \\frac { x ^ { 2 } + 9 } { x - 3 } \\right)"

The optional format_options request parameter allows a request to customize the LaTeX result formats using an object with a format as the property name and the options for that format as the value. The options value may specify the following properties:

Option Type Description
transforms string[] array of transformation names
math_delims [string, string] [begin, end] delimiters for math mode, for example \( and \)
displaymath_delims [string, string] [begin, end] delimiters for displaymath mode, for example \[ and \]

The currently-supported transforms are:

Transform Description
rm_spaces omit spaces around LaTeX groups and other places where spaces are superfluous
rm_newlines uses spaces instead of newlines between text lines in paragraphs
rm_fonts omit mathbb, mathbf, mathcal, and mathrm commands
rm_style_syms replace styled commands with unstyled versions, e.g., bigoplus becomes oplus
rm_text omit text to the left or right of math
long_frac convert longdiv to frac

Note that rm_fonts and rm_style_syms are implicit in latex_normal, latex_simplified, and latex_list. The long_frac transformation is implicit in latex_simplified and latex_list.

Result objects

{
    "detection_list": [],
    "detection_map": {
        "contains_chart": 0,
        "contains_diagram": 0,
        "contains_geometry": 0,
        "contains_graph": 0,
        "contains_table": 0,
        "is_inverted": 0,
        "is_not_math": 0,
        "is_printed": 0
    },
    "latex_normal": "\\lim _ { x \\rightarrow 3 } ( \\frac { x ^ { 2 } + 9 } { x - 3 } )",
    "latex_confidence": 0.86757309488734,
    "position": {
        "height": 273,
        "top_left_x": 57,
        "top_left_y": 14,
        "width": 605
    }
}
Field Type Description
text (optional) string Recognized text format
text_display (optional) string Recognized text_display format
latex_normal (optional) string Recognized latex_normal format
latex_simplified (optional) string Recognized latex_normal format
latex_styled (optional) string Recognized latex_styled format
latex_list (optional) string[] Recognized latex_list format
mathml (optional) string Recognized MathML format
asciimath (optional) string Recognized AsciiMath format
wolfram (optional) string Recognized Wolfram format
position (optional) object Position object, pixel coordinates
detection_list (optional) string[] Detects image properties (see image properties)
error (optional) string US locale error message
error_info (optional) object Error info object
latex_confidence (optional) number in [0,1] Estimated probability 100% correct
candidates (optional) object[] n_best results

Image properties

The API defines multiple detection types:

Detection Definition
contains_diagram Contains a diagram.
is_printed The image is taken of printed math, not handwritten math.
is_not_math No valid equation was detected.

Error info object

In addition to the error field that contains a string (en-us locale) describing an error, a Mathpix response has an error_info field with an object providing programmatic information. The fields of this object include id, uniquely specifying the error, message, containing a string similar to the top-level error field, and detail fields specific to the type of error. The table below lists the different errors Mathpix returns.

Error info fields:

Field Type Description
id string specifies the error id (see below)
message string error message
detail (optional) string Additional error info

Error id types

Here is a table with all the error_id possibilities:

Id Description Detail fields HTTP Status
http_unauthorized Invalid credentials 401
http_max_requests Too many requests count 429
json_syntax JSON syntax error 200
image_missing Missing URL in request body 200
image_download_error Error downloading image url 200
image_decode_error Cannot decode the image data 200
image_no_content No content found in image 200
image_not_supported Image is not math or text 200
image_max_size Image is too large to process 200
opts_bad_callback Bad callback field(s) post?, reply?, batch_id? 200
opts_unknown_ocr Unknown ocr option(s) ocr 200
opts_unknown_format Unknown format option(s) formats 200
opts_number_required Option must be a number name,value 200
opts_value_out_of_range Value not in accepted range name,value 200
math_confidence Low confidence 200
math_syntax Unrecognized math 200
batch_unknown_id Unknown batch id batch_id 200
sys_exception Server error batch_id? 200

Callback request object

Full request body example (includes callback field, shortened src field):

{  
   "src":"data:image/jpeg;base64,...",
   "ocr": ["math", "text"],
   "formats": ["text", "latex_styled"],
   "skip_recrop": true,
   "callback":{  
      "post":"http://RequestBin.com/10z3g561",
      "reply":"integral.jpg"
   }
}
Field Type Description
post string URL to which to make POST callback
headers object key value pairs of headers to make POST
reply (optional) string Sets values of reply field of callback response object (see callback response object)

Callback response object

{
  "reply": "integral.jpg",
  "result": {
    "detection_list": [
      "is_printed"
    ],
    "detection_map": {
      "contains_chart": 0,
      "contains_diagram": 0,
      "contains_geometry": 0,
      "contains_graph": 0,
      "contains_table": 0,
      "is_inverted": 0,
      "is_not_math": 0,
      "is_printed": 1
    },
    "error": "",
    "latex": "\\int \\frac { 4 x } { \\sqrt { x ^ { 2 } + 1 } } d x",
    "latex_confidence": 0.99817252453161,
    "position": {
      "height": 215,
      "top_left_x": 57,
      "top_left_y": 0,
      "width": 605
    }
  }
}
Field Type Description
reply string Request identifier
result object Result object

Process batch (v3/batch)

A batch request is made as follows:

{  
   "urls":{  
      "inverted":"https://raw.githubusercontent.com/Mathpix/api-examples/master/images/inverted.jpg",
      "algebra":"https://raw.githubusercontent.com/Mathpix/api-examples/master/images/algebra.jpg"
   },
   "formats": ["latex_simplified"]
}
curl -X POST https://api.mathpix.com/v3/batch \
-H "app_id: YOUR_APP_ID" \
-H "app_key: YOUR_APP_KEY" \
-H "Content-Type: application/json" \
--data '{ "urls": {"inverted": "https://raw.githubusercontent.com/Mathpix/api-examples/master/images/inverted.jpg", "algebra": "https://raw.githubusercontent.com/Mathpix/api-examples/master/images/algebra.jpg"},"formats":["latex_simplified"] }'

import requests
import json

base_url = 'https://raw.githubusercontent.com/Mathpix/api-examples/master/images/'

data = {
    'urls': {
        'algebra': base_url + 'algebra.jpg',
        'inverted': base_url + 'inverted.jpg'
    },
    'formats': ['latex_simplified']
}

r = requests.post(
    "https://api.mathpix.com/v3/batch", data=json.dumps(data),
    headers={
        'app_id': 'YOUR_APP_ID',
        'app_key': 'YOUR_APP_KEY',
        'content-type': 'application/json'
    },
    timeout=30
)
reply = json.loads(r.text)
assert reply.has_key('batch_id')

The response to the batch is a positive integer value for batch_id.

{
  "batch_id": "17"
}

The response to GET /v3/batch/17 is below when the batch has completed. Before completion the “results” field may be empty or contain only one of the two results.

{
  "keys": ["algebra", "inverted"],
  "results": {
    "algebra": {
      "detection_list": [], 
      "detection_map": {
        "contains_chart": 0, 
        "contains_diagram": 0, 
        "contains_geometry": 0, 
        "contains_graph": 0, 
        "contains_table": 0, 
        "is_inverted": 0, 
        "is_not_math": 0, 
        "is_printed": 0
      }, 
      "latex_simplified": "12 + 5 x - 8 = 12 x - 10", 
      "latex_confidence": 0.99640350138238, 
      "position": {
        "height": 208, 
        "top_left_x": 0, 
        "top_left_y": 0, 
        "width": 1380
      }
    }, 
    "inverted": {
      "detection_list": [
        "is_inverted", 
        "is_printed"
      ], 
      "detection_map": {
        "contains_chart": 0, 
        "contains_diagram": 0, 
        "contains_geometry": 0, 
        "contains_graph": 0, 
        "contains_table": 0, 
        "is_inverted": 1, 
        "is_not_math": 0, 
        "is_printed": 1
      }, 
      "latex_simplified": "x ^ { 2 } + y ^ { 2 } = 9", 
      "latex_confidence": 0.99982263230866, 
      "position": {
        "height": 170, 
        "top_left_x": 48, 
        "top_left_y": 85, 
        "width": 544
      }
    }
  }
}

The Mathpix API supports processing multiple images in a single POST request to a different endpoint: /v3/batch. The request body may contain all the /v3/latex parameters except src and must contain a urls parameter. The request may contain an additonal callback parameter to receive results after all the images in the batch have been processed.

Parameter Type Description
urls object key-value for each image in the batch where the value may be a string url or an object containing a url and image-specific request arguments such as region and formats.
callback (optional) object description of where to send the batch results
callback.post string url to post results
callback.reply (optional) object data to send in reply to batch POST
callback.body (optional) object data to send with results to callback.post
callback.headers (optional) object headers to use when posting results

The response contains only a unique batch_id value. Even if the request includes a callback, there is no guarantee the callback will run successfuly (because of a transient network failure, for example). The preferred approach is to wait an appropriate length of time (about one second for every five images in the batch) and then do a GET on /v3/batch/:id where :id is the batch_id value. The GET request must contain the same app_id and app_key headers as the POST to /v3/batch.

The GET response has the following fields:

Field Type Description
keys string[] all the url keys present in the originating batch request
results object an OCR result for each key that has been processed

Long division

Response for image on the left side:

{
  "detection_map": {
    "contains_chart": 0,
    "contains_diagram": 0,
    "contains_geometry": 0,
    "contains_graph": 0,
    "contains_table": 0,
    "is_inverted": 0,
    "is_not_math": 0,
    "is_printed": 1
  },
  "latex_normal": "8 \\longdiv { 7200 }"
}

The following <script> tags will take care of rendering the \longdiv markup.

<script type="text/javascript" src="//cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/x-mathjax-config">
    MathJax.Hub.Config({
        tex2jax: {inlineMath: [['$','$'], ['\\(','\\)']]},
        messageStyle: "none",
        TeX: {
            Macros: {
                longdiv: ["{\\overline{\\smash{)}#1}}", 1]
            }
        }
    });
</script>

We use the special markup \longdiv to represent long division; it is the only nonvalid Latex markup we return. Long division is used much like \sqrt which is visually similar.

Latency considerations

The biggest source of latency is image uploads. The speed of a response from Mathpix API servers is roughly proportional to the size of the image. Try to use images under 100kb for maximum speeds. JPEG compression and image downsizing are recommended to ensure lowest possible latencies.

Example: Mathpix Snipping Tool

Mathpix Snipping Tool uses the request arguments seen on the right:

{
   "format_options":{
      "text":{
         "displaymath_delims":[
            "\n$$\n",
            "\n$$\n"
         ],
         "transforms":[
            "rm_spaces"
         ],
         "math_delims":[
            "\\(",
            "\\)"
         ]
      },
      "latex_styled":{
         "transforms":[
            "rm_spaces"
         ]
      }
   },
   "skip_recrop":true,
   "ocr":[
      "math",
      "text"
   ],
   "formats":[
      "text",
      "text_display",
      "latex_styled"
   ]
}

Note: src image field is abbreviated to make the JSON more readable.

Vocabulary

Mathpix generates any of the following characters:

! $ &
( ) * + ,
- . / 0 1
2 3 4 5 6
7 8 9 : ;
< = > ? A
B C D E F
G H I J K
L M N O P
Q R S T U
V W X Y Z
[ \ ] ^ _
a b c d e
f g h i j
k l m n o
p q r s t
u v w x y
z { | } ~
\% \\ \{ \}
\alpha \angle \langle \rangle \approx
\because \begin{array} \beta \bot \cap
\cdot \cdots \chi \circ \cong
\cup \dagger \Delta \delta \div
\dot \dots \ell \emptyset \eta
\end{array} \epsilon \equiv \exists \kappa
\forall \frac \gamma \Gamma \geq
\hat \hbar \hline \in \infty
\int \Lambda \lambda \lceil \left(
\left. \left[ \left\{ \left| \Leftrightarrow
\leftrightarrow \lfloor \leq \longdiv \mathcal
\mp \mu \nabla \neq \notin
\oint \Omega \omega \operatorna \oplus
\overline \otimes \parallel \partial \perp
\Phi \phi \pi \pm \prime
\prod \propto \Psi \psi \qquad
\quad \rceil \rfloor \rho \right)
\right. \right\} \right] \Rightarrow \rightarrow
\right| \sigma \sim \simeq \sqrt
\square \star \sum \subset \supset
\subseteq \supseteq \tau \text \therefore
\Theta \theta \times \tilde
\varphi \vdots \vec \wedge \vee
\Xi \xi \zeta

Example outputs