Introduction

git clone git@github.com:Mathpix/api-examples.git
cd api-examples/images

The MathpixOCR API is a JSON API for extracting text from images. It is specialized to work on small to medium images (support for full pages coming soon), and has extensive support for scientific notation, as used in chemistry, math, physics, computer science, economics, and other STEM subjects.

To use MathpixOCR, you must base64 encode your image, and send it to the MathpixOCR API, along with other JSON information.

If you have any problems, please send us an email at support@mathpix.com.

Authorization

content-type: application/json
app_id: YOUR_APP_ID
app_key: YOUR_APP_KEY

MathpixOCR uses API keys to allow access to the API. You can find your API keys on your dashboard, which you can access by logging in or signing up.

MathpixOCR expects for the API key to be included in all API requests to the server via HTTP Basic Auth. Expected set of HTTP headers is shown on the right.

Processing a single image

Request options

You can request as many formats as you wish for a single image:

{
  "src":"data:image/jpeg;base64,...",
  "ocr": ["math", "text"], 
  "skip_recrop": true,
  "formats":[
    "text",
    "latex_simplified",
    "latex_styled",
    "mathml",
    "asciimath",
    "latex_list"
   ]
}

#!/usr/bin/env python
import sys
import base64
import requests
import json

# put desired file path here
file_path = 'limit.jpg'
image_uri = "data:image/jpg;base64," + base64.b64encode(open(file_path, "rb").read())
# (python3) image_uri = "data:image/jpg;base64," + base64.b64encode(open(file_path, "rb").read()).decode()
r = requests.post("https://api.mathpix.com/v3/latex",
    data=json.dumps({'src': image_uri}),
    headers={"app_id": "YOUR_APP_ID", "app_key": "YOUR_APP_KEY",
            "Content-type": "application/json"})
print json.dumps(json.loads(r.text), indent=4, sort_keys=True)

curl -X POST https://api.mathpix.com/v3/latex \
-H 'app_id: YOUR_APP_ID' \
-H 'app_key: YOUR_APP_KEY' \
-H 'Content-Type: application/json' \
--data '{ "src": "data:image/jpeg;base64,'$(base64 -i limit.jpg)'" }'

POST https://api.mathpix.com/v3/latex

Parameter	Type	Description
src	string	Image data, or public URL where image is located
ocr (optional)	[string]	The default is [“math”], to process all text pass [“math”, “text”]
formats	[string]	String postprocessing formats (see Formatting section).
skip_recrop (optional)	bool	Force algorithm to consider whole image
confidence_threshold (optional)	number in [0,1]	Set threshold for triggering confidence errors
region (optional)	object	The region field, if present, specifies the image area to process using the properties top_left_x, top_left_y, width, and height, which are all in pixel coordinates (integer valued).
callback	object	Callback request object

Formatting options

The following formats can be used in the request:

Format	Description
text	to return LaTeX using text mode at the top level instead of math mode,
latex_normal	for the most direct representation of the input
latex_simplified	for modifications that assist symbolic processing such as shortening operator names, replacing long division with a fraction, and converting a column of operands into a single formula
latex_styled	for modifications to improve the visual appearance such as adding ’\left’ and ’\right’ around parenthesized expressions that contain tall expressions like subscript or superscript;
latex_list	to split output into a list of simplified strings that assist in processing multiple equations,
mathml	to return a string containing the MathML for the recognized math; and
wolfram	to return a string that is compatible with the Wolfram Alpha engine.

Result objects

{
    "detection_list": [],
    "detection_map": {
        "contains_chart": 0,
        "contains_diagram": 0,
        "contains_geometry": 0,
        "contains_graph": 0,
        "contains_table": 0,
        "is_inverted": 0,
        "is_not_math": 0,
        "is_printed": 0
    },
    "latex": "\\lim _ { x \\rightarrow 3 } ( \\frac { x ^ { 2 } + 9 } { x - 3 } )",
    "latex_confidence": 0.86757309488734,
    "position": {
        "height": 273,
        "top_left_x": 57,
        "top_left_y": 14,
        "width": 605
    }
}

Field	Type	Description
text (optional)	string	Predicted text format (not shown if not requested as format)
latex (optional)	string	Predicted latex format (not shown if not requested as format)
latex_list (optional)	[string]	Predicted latex_list format
wolfram (optional)	[string]	Predicted wolfram format
asciimath (optional)	[string]	Predicted asciimath format
position (optional)	object	Position object, pixel coordinates
detection_list (optional)	[string]	Detects image properties (see image properties)
error (optional)	string	US locale error message
error_info (optional)	object	Error info object
latex_confidence (optional)	number in [0,1]	Estimated probability 100% correct

Callback request object

Full request body example (includes callback field, shortened src field):

{  
   "src":"data:image/jpeg;base64,...",
   "ocr": ["math", "text"],
   "formats": ["text", "latex_styled"],
   "skip_recrop": true,
   "callback":{  
      "post":"http://RequestBin.com/10z3g561",
      "reply":"integral.jpg"
   }
}

Field	Type	Description
post	string	URL to which to make POST callback
headers	object	key value pairs of headers to make POST
reply (optional)	string	Sets values of reply field of callback response object (see callback response object)

Callback response object

{
  "reply": "integral.jpg",
  "result": {
    "detection_list": [
      "is_printed"
    ],
    "detection_map": {
      "contains_chart": 0,
      "contains_diagram": 0,
      "contains_geometry": 0,
      "contains_graph": 0,
      "contains_table": 0,
      "is_inverted": 0,
      "is_not_math": 0,
      "is_printed": 1
    },
    "error": "",
    "latex": "\\int \\frac { 4 x } { \\sqrt { x ^ { 2 } + 1 } } d x",
    "latex_confidence": 0.99817252453161,
    "position": {
      "height": 215,
      "top_left_x": 57,
      "top_left_y": 0,
      "width": 605
    }
  }
}

Field	Type	Description
reply	string	Request identifier
result	object	Result object

Image properties

The API defines multiple detection types:

Detection	Definition
contains_diagram	Contains a diagram.
is_printed	The image is taken of printed math, not handwritten math.
is_not_math	No valid equation was detected.

Error info object

In addition to the error field that contains a string (en-us locale) describing an error, a Mathpix response has an error_info field with an object providing programmatic information. The fields of this object include id, uniquely specifying the error, message, containing a string similar to the top-level error field, and detail fields specific to the type of error. The table below lists the different errors Mathpix returns.

Error info fields:

Field	Type	Description
id	string	specifies the error id (see below)
message	string	error message
detail (optional)	string	Additional error info

Error id types

Here is a table with all the error_id possibilities:

Id	Description	Detail fields	HTTP Status
http_unauthorized	Invalid credentials		401
http_max_requests	Too many requests	count	429
json_syntax	JSON syntax error		200
image_missing	Missing URL in request body		200
image_download_error	Error downloading image	url	200
image_decode_error	Cannot decode the image data		200
image_no_content	No content found in image		200
image_not_supported	Image is not math or text		200
image_max_size	Image is too large to process		200
opts_bad_callback	Bad callback field(s)	post?, reply?, batch_id?	200
opts_unknown_ocr	Unknown ocr option(s)	ocr	200
opts_unknown_format	Unknown format option(s)	formats	200
math_confidence	Low confidence		200
math_syntax	Unrecognized math		200
batch_unknown_id	Unknown batch id	batch_id	200
sys_exception	Server error	batch_id?	200

Processing a batch of images

A batch request is made as follows:

{  
   "urls":{  
      "inverted":"https://raw.githubusercontent.com/Mathpix/api-examples/master/images/inverted.jpg",
      "algebra":"https://raw.githubusercontent.com/Mathpix/api-examples/master/images/algebra.jpg"
   },
   "callback":{  
      "post":"http://RequestBin.com/sr1x3lsr"
   }
}

curl -X POST https://api.mathpix.com/v3/batch \
-H "app_id: YOUR_APP_ID" \
-H "app_key: YOUR_APP_KEY" \
-H "Content-Type: application/json" \
--data '{ "urls": {"inverted": "https://raw.githubusercontent.com/Mathpix/api-examples/master/images/inverted.jpg", "algebra": "https://raw.githubusercontent.com/Mathpix/api-examples/master/images/algebra.jpg"},"callback":{"post": "http://RequestBin.com/sr1x3lsr"}}'

import requests
import json

base_url = 'https://raw.githubusercontent.com/Mathpix/api-examples/master/images/'

data = {
    'urls': {
        'algebra': base_url + 'algebra.jpg',
        'inverted': base_url + 'inverted.jpg'
    },
    'callback': {
        'post': 'http://RequestBin.com/1ftatkr1'
    }
}

r = requests.post(
    "https://api.mathpix.com/v3/batch", data=json.dumps(data),
    headers={
        'app_id': 'YOUR_APP_ID',
        'app_key': 'YOUR_APP_KEY',
        'content-type': 'application/json'
    },
    timeout=30
)
reply = json.loads(r.text)
assert reply.has_key('batch_id')

The callback response is as follows:

{
  "reply": {
    "batch_id": "11"
  }, 
  "result": {
    "algebra": {
      "detection_list": [], 
      "detection_map": {
        "contains_chart": 0, 
        "contains_diagram": 0, 
        "contains_geometry": 0, 
        "contains_graph": 0, 
        "contains_table": 0, 
        "is_inverted": 0, 
        "is_not_math": 0, 
        "is_printed": 0
      }, 
      "error": "", 
      "latex": "12 + 5 x - 8 = 12 x - 10", 
      "latex_confidence": 0.99640350138238, 
      "latex_list": [], 
      "position": {
        "height": 208, 
        "top_left_x": 0, 
        "top_left_y": 0, 
        "width": 1380
      }
    }, 
    "inverted": {
      "detection_list": [
        "is_inverted", 
        "is_printed"
      ], 
      "detection_map": {
        "contains_chart": 0, 
        "contains_diagram": 0, 
        "contains_geometry": 0, 
        "contains_graph": 0, 
        "contains_table": 0, 
        "is_inverted": 1, 
        "is_not_math": 0, 
        "is_printed": 1
      }, 
      "error": "", 
      "latex": "x ^ { 2 } + y ^ { 2 } = 9", 
      "latex_confidence": 0.99982263230866, 
      "latex_list": [], 
      "position": {
        "height": 170, 
        "top_left_x": 48, 
        "top_left_y": 85, 
        "width": 544
      }
    }
  }
}

The Mathpix API supports processing multiple images in a single POST request to a different endpoint: /v3/batch. The request contains urls specifying a key-url pair for each image, and callback that specifies where to send the results. The url in a key-url pair may either be a string or an object containing the url and additional image-specific request parameters such as region and formats. The response contains a unique batch_id value, and after processing all the images the server posts a message to callback.post containing result with a key-result pair for each image result and reply containing the batch_id.

An application may specify a value for callback.reply in the request to include fields in addition to batch_id in both the response and result post. However, the server-generated batch_id cannot be overridden.

Applications that wish to provide additional data in the post may specify a callback.body value. The server will pass this value to the post. If callback.headers is provided then the server will send the contents as headers in the post.

To check on the status of a batch request an application may use GET /v3/batch/:id where :id is the returned batch_id. Note that the GET request must contain the same app_id and app_key headers as the batch request. The response body is JSON containing the batch url keys (array of strings), current results (key-result objects), and the callback object passed to the original batch request. The server only keeps batch results for a limited amount of time after the batch is complete (currently two days but could change in the future).

Long division

{
    "detection_map": {
        "contains_chart": 0,
        "contains_diagram": 0,
        "contains_geometry": 0,
        "contains_graph": 0,
        "contains_table": 0,
        "is_inverted": 0,
        "is_not_math": 0,
        "is_printed": 1
    },
    "latex": "8 \\longdiv { 7200 }"
}

The following <script> tags will take care of rendering the \longdiv markup.

<script type="text/javascript" src="//cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/x-mathjax-config">
    MathJax.Hub.Config({
        tex2jax: {inlineMath: [['$','$'], ['\\(','\\)']]},
        messageStyle: "none",
        TeX: {
            Macros: {
               longdiv: ["{\\overline{\\smash{)}#1}}", 1]
            }
        }
    });
</script>

We use the special markup \longdiv to represent long division; it is the only nonvalid Latex markup we return. Long division is used much like \sqrt which is visually similar.

Latency considerations

The biggest source of latency is image uploads. The speed of a response from Mathpix API servers is roughly proportional to the size of the image. Try to use images under 100kb for maximum speeds. JPEG compression and image downsizing are recommended to ensure lowest possible latencies.

Example: Mathpix Snipping Tool

Mathpix Snipping Tool uses the JSON options seen on the right:

{
  "src":"data:image/jpeg;base64,...",
  "ocr": ["math", "text"], 
  "skip_recrop": true,
  "formats":[
      {
         "latex":[
            "styled",
            "rm_spaces"
         ]
      },
      {
         "text":[
            "text",
            "rm_spaces"
         ]
      }
   ]
}

Note: src image field is abbreviated to make the JSON more readable.

Vocabulary

Mathpix generates any of the following characters:

!	“	$	&	’
(	)	*	+	,
-	.	/	0	1
2	3	4	5	6
7	8	9	:	;
<	=	>	?	A
B	C	D	E	F
G	H	I	J	K
L	M	N	O	P
Q	R	S	T	U
V	W	X	Y	Z
[	\	]	^	_
a	b	c	d	e
f	g	h	i	j
k	l	m	n	o
p	q	r	s	t
u	v	w	x	y
z	{	|	}	~
\%	\\	\{	\}
\alpha	\angle	\langle	\rangle	\approx
\because	\begin{array}	\beta	\bot	\cap
\cdot	\cdots	\chi	\circ	\cong
\cup	\dagger	\Delta	\delta	\div
\dot	\dots	\ell	\emptyset	\eta
\end{array}	\epsilon	\equiv	\exists	\kappa
\forall	\frac	\gamma	\Gamma	\geq
\hat	\hbar	\hline	\in	\infty
\int	\Lambda	\lambda	\lceil	\left(
\left.	\left[	\left\{	\left|	\Leftrightarrow
\leftrightarrow	\lfloor	\leq	\longdiv	\mathcal
\mp	\mu	\nabla	\neq	\notin
\oint	\Omega	\omega	\operatorna	\oplus
\overline	\otimes	\parallel	\partial	\perp
\Phi	\phi	\pi	\pm	\prime
\prod	\propto	\Psi	\psi	\qquad
\quad	\rceil	\rfloor	\rho	\right)
\right.	\right\}	\right]	\Rightarrow	\rightarrow
\right|	\sigma	\sim	\simeq	\sqrt
\square	\star	\sum	\subset	\supset
\subseteq	\supseteq	\tau	\text	\therefore
\Theta	\theta	\times	\tilde
\varphi	\vdots	\vec	\wedge	\vee
\Xi	\xi	\zeta

Example outputs

• Examples PDF