Introduction

git clone git@github.com:Mathpix/api-examples.git
cd api-examples/images

MathpixOCR recognizes printed and handwritten STEM document content, including math, text, tables, and chemistry diagrams, from an image, stroke data, or a PDF file. The returned content for an image or stroke data may be Mathpix Markdown, LaTeX, AsciiMath, MathML, or HTML. For a PDF file the content is also available as docx.

This page describes the MathpixOCR requests and responses. If you have any questions or problems, please send email to support@mathpix.com.

Authorization

Include the following headers in every request:

{
  "app_id": "APP_ID",
  "app_key": "APP_KEY"
}

Every MathpixOCR request should include two headers: app_id to identify your application and app_key to authorize access to the service. You can find your API key on your account dashboard at https://accounts.mathpix.com/ocr-api.

Process an image

Example image:

JSON request body to send an image URL:

{
  "src": "https://mathpix-ocr-examples.s3.amazonaws.com/cases_hw.jpg",
  "formats": ["text", "data"],
  "data_options": {
    "include_asciimath": true
  }
}

Response:

{
  "auto_rotate_confidence": 0.0046418966250847404,
  "auto_rotate_degrees": 0,
  "confidence": 0.9849104881286621,
  "confidence_rate": 0.9849104881286621,
  "data": [
    {
      "type": "asciimath",
      "value": "f(x)={[x^(2),\" if \"x<0],[2x,\" if \"x>=0]:}"
    }
  ],
  "is_handwritten": true,
  "is_printed": false,
  "request_id": "eec691de4cad35bcf5cfad28d865278b",
  "text": "\\( f(x)=\\left\\{\\begin{array}{ll}x^{2} & \\text { if } x<0 \\\\ 2 x & \\text { if } x \\geq 0\\end{array}\\right. \\)"
}

To process an image with MathpixOCR, a client may post a form to v3/text containing the image file and a single form field options_json specifying the desired options. Alternatively, a client may post a JSON body to v3/text containing a link to the image in the src field along with the desired options. For backward compatibility a client also may post a JSON body containing the base64 encoding of the image in the src field.

The response body is JSON containing the recognized content and information about the recognition. The text field contains Mathpix Markdown, including math as LaTeX inside inline delimiters $ ... $ and block mode delimiters \[ .... \]. Chemistry diagrams are returned as <smiles>...</smiles> where ... is the SMILES (simplified molecular-input line-entry system) representation of chemistry diagrams. Lines are separated with \n newline characters. In some cases (e.g., multiple choice equations) horizontally-aligned content will be flattened into different lines.

We also provide structured data results via the data and html options. The data output returns a list of extracted formats (such as tsv for tables, or asciimath for equations). The html output provides annotated HTML compatible with HTML/XML parsers.

Examples

Send an image URL:

#!/usr/bin/env python
import requests
import json

r = requests.post("https://api.mathpix.com/v3/text",
        json={
            "src": "https://mathpix-ocr-examples.s3.amazonaws.com/cases_hw.jpg",
            "math_inline_delimiters": ["$", "$"],
            "rm_spaces": True
        },
        headers={
            "app_id": "APP_ID",
            "app_key": "APP_KEY",
            "Content-type": "application/json"
        }
    )
print(json.dumps(r.json(), indent=4, sort_keys=True))

curl -X POST https://api.mathpix.com/v3/text \
-H 'app_id: APP_ID' \
-H 'app_key: APP_KEY' \
-H 'Content-Type: application/json' \
--data '{"src": "https://mathpix-ocr-examples.s3.amazonaws.com/cases_hw.jpg", "math_inline_delimiters": ["$", "$"], "rm_spaces": true}'

Send an image file:

#!/usr/bin/env python
import requests
import json

r = requests.post("https://api.mathpix.com/v3/text",
    files={"file": open("cases_hw.jpg","rb")},
    data={
      "options_json": json.dumps({
        "math_inline_delimiters": ["$", "$"],
        "rm_spaces": True
      })
    },
    headers={
        "app_id": "APP_ID",
        "app_key": "APP_KEY"
    }
)
print(json.dumps(r.json(), indent=4, sort_keys=True))

curl -X POST https://api.mathpix.com/v3/text \
-H 'app_id: APP_ID' \
-H 'app_key: APP_KEY' \
--form 'file=@"cases_hw.jpg"' \
--form 'options_json="{\"math_inline_delimiters\": [\"$\", \"$\"], \"rm_spaces\": true}"'

Response

{
  "auto_rotate_confidence": 0.0046418966250847404,
  "auto_rotate_degrees": 0,
  "confidence": 0.9849104881286621,
  "confidence_rate": 0.9849104881286621,
  "is_handwritten": true,
  "is_printed": false,
  "latex_styled": "f(x)=\\left\\{\\begin{array}{ll}\nx^{2} & \\text { if } x<0 \\\\\n2 x & \\text { if } x \\geq 0\n\\end{array}\\right.",
  "request_id": "bd02af63ef187492c085331c60151d98",
  "text": "$f(x)=\\left\\{\\begin{array}{ll}x^{2} & \\text { if } x<0 \\\\ 2 x & \\text { if } x \\geq 0\\end{array}\\right.$"
}

Request parameters

{
  "src": "https://mathpix-ocr-examples.s3.amazonaws.com/cases_hw.jpg",
  "formats": ["text", "data", "html"],
  "data_options": {
    "include_asciimath": true,
    "include_latex": true
  }
}

Parameter	Type	Description
src (optional)	string	Image URL
metadata (optional)	object	Key-value object
tags (optional)	[string]	Tags are lists of strings that can be used to identify results. see query image results
formats (optional)	[string]	List of formats, one of `text`, `data`, `html`, `latex_styled`, see Format Descriptions
data_options (optional)	object	See DataOptions section, specifies outputs for `data` and `html` return fields
include_detected_alphabets (optional)	bool	Return detected alphabets
alphabets_allowed (optional)	object	See AlphabetsAllowed section, use this to specify which alphabets you don't want in the output
confidence_threshold (optional)	number in [0,1]	Specifies threshold for triggering confidence errors
confidence_rate_threshold (optional)	number in [0,1]	Specifies threshold for triggering confidence errors, default 0.75 (symbol level threshold)
include_line_data (optional)	bool	Specifies whether to return information segmented line by line, see LineData object section for details
include_word_data (optional)	bool	Specifies whether to return information segmented word by word, see WordData object section for details
include_smiles (optional)	bool	Enable experimental chemistry diagram OCR, via RDKIT normalized SMILES with `isomericSmiles=False`, included in `text` output format, via MMD SMILES syntax `<smiles>...</smiles>`
include_inchi (optional)	bool	Include InChI data as XML attributes inside `<smiles>` elements, for examples `<smiles inchi="..." inchikey="...">...</smiles>`; only applies when `include_smiles` is true
include_geometry_data (optional)	bool	Enable data extraction for geometry diagrams (currently only supports triangle diagrams); see GeometryData
auto_rotate_confidence_threshold (optional)	number in [0,1]	Specifies threshold for auto rotating image to correct orientation; by default it is set to 0.99, can be disabled with a value of 1 (see Auto rotation section for details)
rm_spaces (optional)	bool	Determines whether extra white space is removed from equations in `latex_styled` and `text` formats. Default is `true`.
rm_fonts (optional)	bool	Determines whether font commands such as \mathbf and \mathrm are removed from equations in `latex_styled` and `text` formats. Default is `false`.
idiomatic_eqn_arrays (optional)	bool	Specifies whether to use aligned, gathered, or cases instead of an array environment for a list of equations. Default is `false`.
numbers_default_to_math (optional)	bool	Specifies whether numbers are always math, e.g., `Answer: $ 17 $` instead of `Answer: 17`. Default is `false`.
math_inline_delimiters (optional)	[string, string]	Specifies begin inline math and end inline math delimiters for `text` outputs. Default is `["\$", "\$"]`.
math_display_delimiters (optional)	[string, string]	Specifies begin display math and end display math delimiters for `text` outputs. Default is `["\\[", "\\]"]`.

Format descriptions

MathpixOCR returns strings in one of the selected formats:

Format	Description
text	Mathpix Markdown
html	HTML rendered from `text` via mathpix-markdown-it
data	Data computed from `text` as specified in the `data_options` request parameter
latex_styled	Styled Latex, returned only in cases that the whole image can be reduced to a single equation

DataOptions object

Data options are used to return elements of the image output. These outputs are all computed from the text format described above.

Key	Type	Description
include_svg (optional)	bool	include math SVG in `html` and `data` formats
include_table_html (optional)	bool	include HTML for `html` and `data` outputs (tables only)
include_latex (optional)	bool	include math mode latex in `data` and `html`
include_tsv (optional)	bool	include tab separated values (TSV) in `data` and `html` outputs (tables only)
include_asciimath (optional)	bool	include asciimath in `data` and `html` outputs
include_mathml (optional)	bool	include mathml in `data` and `html` outputs

Response body

Get an API response:

{
  "confidence": 0.9982182085336344,
  "confidence_rate": 0.9982182085336344,
  "is_printed": false,
  "is_handwritten": true,
  "data": [
    {
      "type": "asciimath",
      "value": "lim_(x rarr3)((x^(2)+9)/(x-3))"
    },
    {
      "type": "latex",
      "value": "\\lim _{x \\rightarrow 3}\\left(\\frac{x^{2}+9}{x-3}\\right)"
    }
  ],
  "html": "<div><span  class=\"math-inline \" >\n<asciimath style=\"display: none;\">lim_(x rarr3)((x^(2)+9)/(x-3))</asciimath><latex style=\"display: none\">\\lim _{x \\rightarrow 3}\\left(\\frac{x^{2}+9}{x-3}\\right)</latex></span></div>\n",
  "text": "\\( \\lim _{x \\rightarrow 3}\\left(\\frac{x^{2}+9}{x-3}\\right) \\)"
}

Field	Type	Description
request_id	string	Request ID, for debugging purposes
text (optional)	string	Recognized `text` format, if such is found
latex_styled (optional)	string	Math Latex string of math equation, if the image is of a single equation
confidence (optional)	number in [0,1]	Estimated probability 100% correct
confidence_rate (optional)	number in [0,1]	Estimated confidence of input quality
line_data (optional)	[object]	List of LineData objects
word_data (optional)	[object]	List of WordData objects
data (optional)	[object]	List of Data objects
html (optional)	string	Annotated HTML output
detected_alphabets (optional)	[object]	DetectedAlphabet object
is_printed (optional)	bool	Specifies if printed content was detected in an image
is_handwritten (optional)	bool	Specifies if handwritten content was detected in an image
auto_rotate_confidence (optional)	number in [0,1]	Estimated probability that image needs to be rotated, see Auto rotation
geometry_data (optional)	[object]	List of GeometryData objects
auto_rotate_degrees (optional)	number in {0, 90, -90, 180}	Estimated angle of rotation in degrees to put image in correct orientation, see Auto rotation
error (optional)	string	US locale error message
error_info (optional)	object	Error info object

Data object

Data objects allow extracting the math elements from an OCR result.

Field	Type	Description
type	string	one of `asciimath`, `mathml`, `latex`, `svg`, `tsv`
value	string	value corresponding to `type`

DetectedAlphabet object

The detected_alphabets object in a result contains a field that is true of false for each known alphabet. The field is true if any characters from the alphabet are recognized in the image, regardless of whether any of the result fields contain the characters.

Field	Type	Description
en	bool	English
hi	bool	Hindi Devenagari
zh	bool	Chinese
ja	bool	Kana Hiragana or Katakana
ko	bool	Hangul Jamo
ru	bool	Russian
th	bool	Thai

AlphabetsAllowed object

{
  "formats": ["text"],
  "alphabets_allowed": {
    "hi": false,
    "zh": false,
    "ja": false,
    "ko": false,
    "ru": false,
    "th": false
  }
}

There are cases where it is not easy to infer the correct alphabet for a single letter because there are different letters from different alphabets that look alike. To illustrate, one example is conflict between Latin B and Cyrillic В (that is Latin V). While being displayed almost the same, they essentially have different Unicode encodings. The option alphabets_allowed can be used to specify map from string to boolean values which can be used to prevent symbols from unwanted alphabet appearing in the result. Map keys that are valid correspond to the values in Field column of the table specified in Detected alphabet object section (e.g. hi or ru). By default all alphabets are allowed in the output, to disable alphabet specify "alphabets_allowed": {"alphabet_key": false}.

Specifying "alphabets_allowed": {"alphabet_key": true} has the same effect as not specifying that alphabet inside alphabets_allowed map.

LineData object

Example request:

{
  "src": "https://mathpix.com/examples/text_with_diagram.png",
  "formats": ["text"],
  "include_line_data": true
}

JSON response:

{
  "confidence": 0.651358435330524,
  "confidence_rate": 0.651358435330524,
  "text": "Equivalent resistance between points \\( \\mathrm{A} \\& \\mathrm{B} \\) in the adjacent circuit is",
  "line_data": [
    {
      "type": "text",
      "cnt": [
        [859, 81],
        [739, 91],
        [626, 91],
        [-2, 66],
        [0, 34],
        [739, 52],
        [859, 63]
      ],
      "included": true,
      "text": "Equivalent resistance between points \\( \\mathrm{A} \\& \\mathrm{B} \\) in the adjacent circuit is",
      "after_hyphen": false,
      "confidence": 0.651358435330524,
      "confidence_rate": 0.9948483133235457
    },
    {
      "type": "diagram",
      "cnt": [
        [654, 244],
        [651, 683],
        [7, 678],
        [11, 238]
      ],
      "included": false,
      "error_id": "image_not_supported"
    }
  ]
}

Field	Type	Description
type	string	One of `text`, `math`, `table`, `diagram`, `equation_number`, `diagram_label`
subtype (optional)	string	Either not set, or `chemistry`, or `triangle` (more diagram subtypes coming soon)
cnt	[[x,y]]	Countour for line expressed as list of (x,y) pixel coordinate pairs
included	bool	Whether this line is included in the top level OCR result
error_id (optional)	string	Error ID, reason why the line is not included in final result
text (optional)	string	Text (Mathpix Markdown) for line
confidence (optional)	number in [0,1]	Estimated probability 100% correct
confidence_rate (optional)	number in [0,1]	Estimated confidence of input quality
after_hyphen (optional)	bool	specifies if the current line occurs after the text line which ended with hyphen
html (optional)	string	Annotated HTML output for the line
data (optional)	[Data]	List of Data object's

Specifying include_line_data to be true will add a line_data field to the response body. This field is a list of LineData objects that contain information about all textual line elements detected in the image. Simply concatenating information from the response's line_data is enough to recreate the top level text, html, and data fields in the response body.

The OCR engine does not support some lines, like diagrams, and these lines are therefore simply skipped. Some lines contain content that is most likely extraneous, like equation numbers. Additionally, sometimes the OCR engine simply cannot recognize the line with proper confidence. In all those cases included field is set to false, meaning the line is not part of the final result.

A line can have the following values for error_id:

image_not_supported - OCR engine doesn't accept this type of line
image_max_size - line is larger than maximal size which OCR engine supports
math_confidence - OCR engine failed to confidently recognize the content of the line
image_no_content - line has strange spatial dimensions, e.g. height of the line is zero; this error is very unlikely to happen

WordData object

Example request:

{
  "src": "https://mathpix.com/examples/text_with_math_0.jpg",
  "include_word_data": true
}

JSON response:

{
  "is_printed": true,
  "is_handwritten": false,
  "auto_rotate_confidence": 0.00939574267408716,
  "auto_rotate_degrees": 0,
  "word_data": [
    {
      "type": "text",
      "cnt": [
        [111, 104],
        [3, 104],
        [3, 74],
        [111, 74]
      ],
      "text": "Perform",
      "confidence": 0.99951171875,
      "confidence_rate": 0.9999593007867263,
      "latex": "\\text { Perform }"
    },
    {
      "type": "text",
      "cnt": [
        [160, 104],
        [115, 104],
        [115, 74],
        [160, 74]
      ],
      "text": "the",
      "confidence": 1,
      "confidence_rate": 1,
      "latex": "\\text { the }"
    },
    {
      "type": "text",
      "cnt": [
        [286, 104],
        [163, 104],
        [163, 74],
        [286, 74]
      ],
      "text": "indicated",
      "confidence": 0.9970722198486328,
      "confidence_rate": 0.9997905880380586,
      "latex": "\\text { indicated }"
    },
    {
      "type": "text",
      "cnt": [
        [413, 107],
        [290, 107],
        [290, 77],
        [413, 77]
      ],
      "text": "operation",
      "confidence": 0.9985356330871582,
      "confidence_rate": 0.9998953311820248,
      "latex": "\\text { operation }"
    },
    {
      "type": "text",
      "cnt": [
        [469, 104],
        [417, 104],
        [417, 74],
        [469, 74]
      ],
      "text": "and",
      "confidence": 1,
      "confidence_rate": 1,
      "latex": "\\text { and }"
    },
    {
      "type": "text",
      "cnt": [
        [592, 108],
        [472, 108],
        [472, 74],
        [592, 74]
      ],
      "text": "simplify.",
      "confidence": 0.9790234565734863,
      "confidence_rate": 0.9984868832735626,
      "latex": "\\text { simplify. }"
    },
    {
      "type": "text",
      "cnt": [
        [126, 162],
        [100, 162],
        [100, 130],
        [126, 130]
      ],
      "text": "3)",
      "confidence": 0.998046875,
      "confidence_rate": 0.9997207483071853,
      "latex": "\\text { 3) }"
    },
    {
      "type": "math",
      "cnt": [
        [322, 191],
        [132, 191],
        [132, 110],
        [322, 110]
      ],
      "text": "\\( \\frac{2 p-2}{p} \\div \\frac{4 p-4}{9 p^{2}} \\)",
      "confidence": 0.99853515625,
      "confidence_rate": 0.9999436201400773,
      "latex": "\\frac{2 p-2}{p} \\div \\frac{4 p-4}{9 p^{2}}"
    }
  ]
}

Field	Type	Description
type	string	One of `text`, `math`, `table`, `diagram`, `equation_number`
subtype (optional)	string	Either not set, or `chemistry`, or `triangle` (more diagram subtypes coming soon)
cnt	[[x,y]]	Countour for word expressed as list of (x,y) pixel coordinate pairs
text (optional)	string	Text (Mathpix Markdown) for word
latex (optional)	string	Math mode LaTeX (Mathpix Markdown) for word
confidence (optional)	number in [0,1]	Estimated probability 100% correct
confidence_rate (optional)	number in [0,1]	Estimated confidence of input quality

Specifying true for include_word_data will add a word_data field to the response body. This field is a a list of WordData objects containing information about all word level elements detected in the image.

Auto rotation

Sometimes images that are received are in wrong orientation,one example of such image would look like this:

The goal of automatic rotation is to pick correct orientation for received images before any processing is done. The result of auto rotation looks like:

One can control how confident our algorithm needs to be to perform auto rotation with request parameter auto_rotate_confidence_threshold, a number in [0,1]. By default the value 0.99 is used, which essentially means that algorithm will auto rotate image if it is 99% confident in that decision. Auto rotation can be disabled by specifying "auto_rotate_confidence_threshold": 1 as a part of request being sent.

The response body will include two related fields:

auto_rotate_confidence - confidence of the algorithm in decision that image needs to be rotated, number in [0,1] which should be ~0 if image is in correct orientation and ~1 if image should be rotated; monitoring this value on custom workloads can be helpful in determining proper auto_rotate_confidence_threshold to be used with such workloads
auto_rotate_degrees - angle in degrees specifying how much to rotate original image to get image in proper orientation, number in {0, 90, -90, 180}, value 0 means image is already in correct orientation; this information can be used to put image in right orientation for other parts of custom processing pipelines

Geometry objects

Example request to v3/text:

{
  "src": "https://mathpix.com/examples/triangle_0.jpg",
  "include_geometry_data": true
}

JSON response:

{
  "is_printed": true,
  "is_handwritten": false,
  "auto_rotate_confidence": 0.03240217728347261,
  "auto_rotate_degrees": 0,
  "geometry_data": [
    {
      "position": {
        "top_left_x": 183,
        "top_left_y": 3,
        "height": 344,
        "width": 309
      },
      "shape_list": [
        {
          "type": "triangle",
          "vertex_list": [
            {
              "x": 218,
              "y": 46,
              "edge_list": [1, 2]
            },
            {
              "x": 218,
              "y": 314,
              "edge_list": [0, 2]
            },
            {
              "x": 456,
              "y": 314,
              "edge_list": [0, 1]
            }
          ]
        }
      ],
      "label_list": [
        {
          "position": {
            "top_left_x": 198,
            "top_left_y": 7,
            "height": 24,
            "width": 22
          },
          "text": "\\( A \\)",
          "latex": "A",
          "confidence": 0.99951171875,
          "confidence_rate": 0.99951171875
        },
        {
          "position": {
            "top_left_x": 228,
            "top_left_y": 78,
            "height": 20,
            "width": 14
          },
          "text": "\\( ? \\)",
          "latex": "?",
          "confidence": 0.96923828125,
          "confidence_rate": 0.96923828125
        },
        {
          "position": {
            "top_left_x": 349,
            "top_left_y": 154,
            "height": 21,
            "width": 16
          },
          "text": "6",
          "latex": "6",
          "confidence": 0.99951171875,
          "confidence_rate": 0.99951171875
        },
        {
          "position": {
            "top_left_x": 189,
            "top_left_y": 318,
            "height": 24,
            "width": 23
          },
          "text": "\\( C \\)",
          "latex": "C",
          "confidence": 0.837890625,
          "confidence_rate": 0.837890625
        },
        {
          "position": {
            "top_left_x": 329,
            "top_left_y": 326,
            "height": 20,
            "width": 18
          },
          "text": "4",
          "latex": "4",
          "confidence": 1,
          "confidence_rate": 1
        },
        {
          "position": {
            "top_left_x": 469,
            "top_left_y": 311,
            "height": 22,
            "width": 22
          },
          "text": "\\( B \\)",
          "latex": "B",
          "confidence": 0.99072265625,
          "confidence_rate": 0.99072265625
        }
      ]
    }
  ]
}

Geometry data is requested via the include_geometry_data request parameter. Currently, only triangle diagrams are supported. Geometry data structures are represented via vertices, edges, and labels.

GeometryData object

The geometry_data object contains the following parameters:

Field	Type	Description
position (optional)	object	Position object, pixel coordinates
shape_list	[object]	List of ShapeData objects
label_list	[object]	List of LabelData objects

ShapeData object

Field	Type	Description
type	string	Type of diagram; currently only `triangle` is supported
vertex_list	[object]	List of VertexData objects

VertexData object

Field	Type	Description
x	integer	x-pixel coordinate for the vertex, counting from top left
y	integer	y-pixel coordinate for the vertex, counting from top left
edge_list	[integer]	List of indices the vertex is connected to, in ShapeData.vertex_list (0 based indexing is used)

LabelData object

Field	Type	Description
position	object	Position object, pixel coordinates
text	string	`text` output for OCR-ed label
latex	string	`latex` output for OCR-ed label
confidence (optional)	number in [0,1]	Estimated probability 100% correct
confidence_rate (optional)	number in [0,1]	Estimated confidence of input quality

Process an equation image

Example image:

JSON request body to send an image URL:

{
  "src": "https://mathpix-ocr-examples.s3.amazonaws.com/limit.jpg",
  "formats": ["latex_normal"]
}

Response:

{
  "auto_rotate_confidence": 0.0006091455863810324,
  "auto_rotate_degrees": 0,
  "detection_list": [],
  "detection_map": {
    "contains_chart": 0,
    "contains_diagram": 0,
    "contains_graph": 0,
    "contains_table": 0,
    "is_blank": 0,
    "is_inverted": 0,
    "is_not_math": 0,
    "is_printed": 5.874842099728994e-5
  },
  "latex_confidence": 1,
  "latex_confidence_rate": 1,
  "latex_normal": "\\operatorname { lim } _ { x \\rightarrow 3 } ( \\frac { x ^ { 2 } + 9 } { x - 3 } )",
  "position": {
    "height": 263,
    "top_left_x": 59,
    "top_left_y": 25,
    "width": 578
  },
  "request_id": "85c5aa804c0db262ec806967acba0999"
}

An equation image is a relatively small image that is mostly math. Applications can use v3/text or v3/text and pass data_options with the desired math format to examine only the math in an image, but for convenience a client also may post a form to v3/latex containing the image file and the form field options_json with the desired options. Alternatively, a client may post a JSON body to v3/latex containing a link to the image in the src field along with the desired options. For backward compatibility a client also may post a JSON body containing the base64 encoding of the image in the src field.

Examples

Send an URL

#!/usr/bin/env python
import requests
import json

data = {
    "src": "https://mathpix-ocr-examples.s3.amazonaws.com/limit.jpg",
    "formats": [
        "latex_simplified",
        "asciimath"
    ]
}
r = requests.post("https://api.mathpix.com/v3/latex",
    json=data,
    headers={
        "app_id": "APP_ID",
        "app_key": "APP_KEY",
        "Content-type": "application/json"
    }
)
print(json.dumps(r.json(), indent=4, sort_keys=True))

curl -X POST https://api.mathpix.com/v3/latex \
-H 'app_id: APP_ID' \
-H 'app_key: APP_KEY' \
-H 'Content-Type: application/json' \
--data '{"src": "https://mathpix-ocr-examples.s3.amazonaws.com/limit.jpg", "formats": ["latex_simplified", "asciimath"]}'

Send an image file:

#!/usr/bin/env python
import requests
import json

r = requests.post("https://api.mathpix.com/v3/latex",
    files={"file": open("limit.jpg","rb")},
    data={"options_json": json.dumps({
        "formats": ["latex_simplified", "asciimath"]
    })},
    headers={
        "app_id": "APP_ID",
        "app_key": "APP_KEY"
    }
)
print(json.dumps(r.json(), indent=4, sort_keys=True))

curl -X POST https://api.mathpix.com/v3/latex \
-H 'app_id: APP_ID' \
-H 'app_key: APP_KEY' \
--form 'file=@"limit.jpg"' \
--form 'options_json="{\"formats\": [\"latex_simplified\", \"asciimath\"]}"'

{
  "asciimath": "lim_(x rarr3)((x^(2)+9)/(x-3))",
  "auto_rotate_confidence": 0.0006091455863810324,
  "auto_rotate_degrees": 0,
  "detection_list": [],
  "detection_map": {
    "contains_chart": 0,
    "contains_diagram": 0,
    "contains_graph": 0,
    "contains_table": 0,
    "is_blank": 0,
    "is_inverted": 0,
    "is_not_math": 0,
    "is_printed": 5.874842099728994e-5
  },
  "latex_confidence": 1,
  "latex_confidence_rate": 1,
  "latex_simplified": "\\lim _ { x \\rightarrow 3 } ( \\frac { x ^ { 2 } + 9 } { x - 3 } )",
  "position": {
    "height": 263,
    "top_left_x": 59,
    "top_left_y": 25,
    "width": 578
  },
  "request_id": "aab527abf5c78bd3a07c51804bcfbfaa"
}

Request parameters

You can request multiple formats for a single image:

{
  "ocr": ["math", "text"],
  "skip_recrop": true,
  "formats": [
    "text",
    "latex_simplified",
    "latex_styled",
    "mathml",
    "asciimath",
    "latex_list"
  ]
}

Parameter	Type	Description
src (optional)	string	Image URL
tags (optional)	[string]	Tags are lists of strings that can be used to identify results. see OCR Results
formats	string[]	String postprocessing formats (see Formatting section)
ocr (optional)	string[]	Process only math ["math"] or both math and text ["math", "text"]
format_options (optional)	object	Options for specific formats (see Formatting section)
skip_recrop (optional)	bool	Force algorithm to consider whole image
confidence_threshold (optional)	number in [0,1]	Set threshold for triggering confidence errors
beam_size (optional)	number in [1,5]	Number of results to consider during recognition
n_best (optional)	number in [1,beam_size]	Number of highest-confidence results to return
region (optional)	object	Specify the image area with the pixel coordinates `top_left_x`, `top_left_y`, `width`, and `height`
callback (optional)	object	Callback request object
metadata (optional)	object	Key value object
include_detected_alphabets (optional)	bool	Return detected alphabets
auto_rotate_confidence_threshold (optional)	number in [0,1]	Specifies threshold for auto rotating image to correct orientation; by default it is set to 0.99, can be disabled with a value of 1 (see Auto rotation section for details)

Formatting

The following formats can be used in the request:

Format	Description
text	text mode output, with math inside delimiters, eg. `test $x^2$`, inline math by default
text_display	same as `text`, except uses block mode math instead of inline mode when in doubt
latex_normal	direct LaTeX representation of the input
latex_styled	modified output to improve the visual appearance such as adding '\left' and '\right' around parenthesized expressions that contain tall expressions like subscript or superscript
latex_simplified	modified output for symbolic processing such as shortening operator names, replacing long division with a fraction, and converting a column of operands into a single formula
latex_list	output split into a list of simplified strings to help process multiple equations
mathml	the MathML for the recognized math
asciimath	the AsciiMath for the recognized math
wolfram	a string compatible with the Wolfram Alpha engine

Format options

To return a more compact latex_styled result, one could send the following request:

{
  "ocr": ["math", "text"],
  "skip_recrop": true,
  "formats": [
    "text",
    "latex_simplified",
    "latex_styled",
    "mathml",
    "asciimath",
    "latex_list"
  ],
  "format_options": {
    "latex_styled": { "transforms": ["rm_spaces"] }
  }
}

The result for "latex*styled" would now be
> "\\lim*{x \\rightarrow 3}\\left(\\frac{x^{2}+9}{x-3}\\right)"
instead of
> "\\lim \_ { x \\rightarrow 3 } \\left( \\frac { x ^ { 2 } + 9 } { x - 3 } \\right)"

The optional format_options request parameter allows a request to customize the LaTeX result formats using an object with a format as the property name and the options for that format as the value. The options value may specify the following properties:

Option	Type	Description
transforms	string[]	array of transformation names
math_delims	[string, string]	[begin, end] delimiters for math mode, for example `$` and `$`
displaymath_delims	[string, string]	[begin, end] delimiters for displaymath mode, for example `\[` and `\]`

The currently-supported transforms are:

Transform	Description
rm_spaces	omit spaces around LaTeX groups and other places where spaces are superfluous
rm_newlines	uses spaces instead of newlines between text lines in paragraphs
rm_fonts	omit mathbb, mathbf, mathcal, and mathrm commands
rm_style_syms	replace styled commands with unstyled versions, e.g., bigoplus becomes oplus
rm_text	omit text to the left or right of math
long_frac	convert longdiv to frac

Note that rm_fonts and rm_style_syms are implicit in latex_normal, latex_simplified, and latex_list. The long_frac transformation is implicit in latex_simplified and latex_list.

Response body

{
  "detection_list": [],
  "detection_map": {
    "contains_chart": 0,
    "contains_diagram": 0,
    "contains_geometry": 0,
    "contains_graph": 0,
    "contains_table": 0,
    "is_inverted": 0,
    "is_not_math": 0,
    "is_printed": 0
  },
  "latex_normal": "\\lim _ { x \\rightarrow 3 } ( \\frac { x ^ { 2 } + 9 } { x - 3 } )",
  "latex_confidence": 0.86757309488734,
  "latex_confidence_rate": 0.9875550770759583,
  "position": {
    "height": 273,
    "top_left_x": 57,
    "top_left_y": 14,
    "width": 605
  }
}

Field	Type	Description
text (optional)	string	Recognized `text` format
text_display (optional)	string	Recognized `text_display` format
latex_normal (optional)	string	Recognized `latex_normal` format
latex_simplified (optional)	string	Recognized `latex_normal` format
latex_styled (optional)	string	Recognized `latex_styled` format
latex_list (optional)	string[]	Recognized `latex_list` format
mathml (optional)	string	Recognized MathML format
asciimath (optional)	string	Recognized AsciiMath format
wolfram (optional)	string	Recognized Wolfram format
position (optional)	object	Position object, pixel coordinates
detection_list (optional)	string[]	Detects image properties (see image properties)
error (optional)	string	US locale error message
error_info (optional)	object	Error info object
latex_confidence (optional)	number in [0,1]	Estimated probability 100% correct
latex_confidence_rate (optional)	number in [0,1]	Estimated confidence of input quality
candidates (optional)	object[]	n_best results
detected_alphabets (optional)	[object]	DetectedAlphabet object
auto_rotate_confidence (optional)	number in [0,1]	Estimated probability that image needs to be rotated, see Auto rotation
auto_rotate_degrees (optional)	number in {0, 90, -90, 180}	Estimated angle of rotation in degrees to put image in correct orientation, see Auto rotation

The detected_alphabets result object contains a field that is true of false for each known alphabet. The field is true if any characters from the alphabet are recognized in the image, regardless of whether any of the result fields contain the characters.

Field	Type	Description
en	bool	English
hi	bool	Hindi Devenagari
zh	bool	Chinese
ja	bool	Kana Hiragana or Katakana
ko	bool	Hangul Jamo
ru	bool	Russian
th	bool	Thai

Image properties

The API defines multiple detection types:

Detection	Definition
contains_diagram	Contains a diagram.
is_printed	The image is taken of printed math, not handwritten math.
is_not_math	No valid equation was detected.

Process stroke data

To process strokes coordinates, a client may post a JSON body to v3/strokes containing the strokes data along with the same options available when processing an image.

This endpoint is very convenient for users that were generating images of handwritten math and text and then using the service v3/text, since with v3/strokes no image generation is required, the request payload is smaller and therefore it results in faster response time.

The LaTeX of the recognized handwriting is returned inside inline delimiters $ ... $ and block mode delimiters \[ .... \]. Lines are separated with \n newline characters. In some cases (e.g. multiple choice equations) we will try to flatten horizontally aligned content into different lines in order to keep the markup simple.

Request parameters

Send some strokes:

{
  "strokes": {
    "strokes": {
      "x": [
        [
          131, 131, 130, 130, 131, 133, 136, 146, 151, 158, 161, 162, 162, 162,
          162, 159, 155, 147, 142, 137, 136, 138, 143, 160, 171, 190, 197, 202,
          202, 202, 201, 194, 189, 177, 170, 158, 153, 150, 148
        ],
        [231, 231, 233, 235, 239, 248, 252, 260, 264, 273, 277, 280, 282, 283],
        [
          273, 272, 271, 270, 267, 262, 257, 249, 243, 240, 237, 235, 234, 234,
          233, 233
        ],
        [
          296, 296, 297, 299, 300, 301, 301, 302, 303, 304, 305, 306, 306, 305,
          304, 298, 294, 286, 283, 281, 281, 282, 284, 284, 285, 287, 290, 293,
          294, 299, 301, 308, 309, 314, 315, 316
        ]
      ],
      "y": [
        [
          213, 213, 212, 211, 210, 208, 207, 206, 206, 209, 212, 217, 220, 227,
          230, 234, 236, 238, 239, 239, 239, 239, 239, 239, 241, 247, 252, 259,
          261, 264, 266, 269, 270, 271, 271, 271, 270, 269, 268
        ],
        [231, 231, 232, 235, 238, 246, 249, 257, 261, 267, 270, 272, 273, 274],
        [
          230, 230, 230, 231, 234, 240, 246, 258, 268, 273, 277, 281, 281, 283,
          283, 284
        ],
        [
          192, 192, 191, 189, 188, 187, 187, 187, 188, 188, 190, 193, 195, 198,
          200, 205, 208, 213, 215, 215, 215, 214, 214, 214, 214, 216, 218, 220,
          221, 223, 223, 223, 223, 221, 221, 220
        ]
      ]
    }
  }
}

#!/usr/bin/env python
import requests
import json

# put input strokes here
strokes_string = '{"strokes": {\
    "x": [[131,131,130,130,131,133,136,146,151,158,161,162,162,162,162,159,155,147,142,137,136,138,143,160,171,190,197,202,202,202,201,194,189,177,170,158,153,150,148],[231,231,233,235,239,248,252,260,264,273,277,280,282,283],[273,272,271,270,267,262,257,249,243,240,237,235,234,234,233,233],[296,296,297,299,300,301,301,302,303,304,305,306,306,305,304,298,294,286,283,281,281,282,284,284,285,287,290,293,294,299,301,308,309,314,315,316]],\
    "y": [[213,213,212,211,210,208,207,206,206,209,212,217,220,227,230,234,236,238,239,239,239,239,239,239,241,247,252,259,261,264,266,269,270,271,271,271,270,269,268],[231,231,232,235,238,246,249,257,261,267,270,272,273,274],[230,230,230,231,234,240,246,258,268,273,277,281,281,283,283,284],[192,192,191,189,188,187,187,187,188,188,190,193,195,198,200,205,208,213,215,215,215,214,214,214,214,216,218,220,221,223,223,223,223,221,221,220]]\
  }}'
strokes = json.loads(strokes_string)
r = requests.post("https://api.mathpix.com/v3/strokes",
    json={"strokes": strokes},
    headers={"app_id": "APP_ID", "app_key": "APP_KEY",
             "Content-type": "application/json"})
print(json.dumps(r.json(), indent=4, sort_keys=True))

curl -X POST https://api.mathpix.com/v3/strokes \
-H 'app_id: APP_ID' \
-H 'app_key: APP_KEY' \
-H 'Content-Type: application/json' \
--data '{ "strokes": {"strokes": {
  "x": [[131,131,130,130,131,133,136,146,151,158,161,162,162,162,162,159,155,147,142,137,136,138,143,160,171,190,197,202,202,202,201,194,189,177,170,158,153,150,148],[231,231,233,235,239,248,252,260,264,273,277,280,282,283],[273,272,271,270,267,262,257,249,243,240,237,235,234,234,233,233],[296,296,297,299,300,301,301,302,303,304,305,306,306,305,304,298,294,286,283,281,281,282,284,284,285,287,290,293,294,299,301,308,309,314,315,316]],
  "y": [[213,213,212,211,210,208,207,206,206,209,212,217,220,227,230,234,236,238,239,239,239,239,239,239,241,247,252,259,261,264,266,269,270,271,271,271,270,269,268],[231,231,232,235,238,246,249,257,261,267,270,272,273,274],[230,230,230,231,234,240,246,258,268,273,277,281,281,283,283,284],[192,192,191,189,188,187,187,187,188,188,190,193,195,198,200,205,208,213,215,215,215,214,214,214,214,216,218,220,221,223,223,223,223,221,221,220]]
  }}}'

Parameter	Type	Description
strokes	JSON	Strokes in JSON with appropriate format.

All other Request Params from v3/text are supported, See Request parameters

Response body

Get an API response:

{
  "request_id": "cea6b8e40ab4550ac467ce2eb00430be",
  "is_printed": false,
  "is_handwritten": true,
  "auto_rotate_confidence": 0.0020149118193977245,
  "auto_rotate_degrees": 0,
  "confidence": 1,
  "confidence_rate": 1,
  "latex_styled": "3 x^{2}",
  "text": "\\( 3 x^{2} \\)"
}

Field	Type	Description
text (optional)	string	Recognized `text` format, if such is found
confidence (optional)	number in [0,1]	Estimated probability 100% correct
confidence_rate (optional)	number in [0,1]	Estimated confidence of input quality
data (optional)	[object]	List of data objects (see "Data object" section above)
html (optional)	string	Annotated HTML output

Query image results

Mathpix allows customers to search their results from posts to /v3/text, /v3/strokes, and /v3/latex with a GET request on /v3/ocr-results?search-parameters.

The search request must also be authenticated the same way OCR requests are, in order to return relevant search results. See Authorization

Requests with the metadata improve_mathpix field set to false will not appear in the search results.

Note that this endpoint will only work with API keys created after July 5th, 2020.

Request parameters

curl -X GET -H 'app_key: APP_KEY' \
    'https://api.mathpix.com/v3/ocr-results?from_date=2021-07-02T18%3A48%3A25.769285%2B00%3A00&page=1&per_page=10&tags=test_tag2&tags=test_tag3&is_handwritten=True'

{
  "ocr_results": [
    {
      "timestamp": "2021-07-02T18:48:46.080Z",
      "duration": 0.132,
      "endpoint": "/v3/text",
      "request_args": {
        "tags": ["test_tag2", "test_tag3"],
        "formats": ["text"]
      },
      "result": {
        "text": "\\( 12+5 x-8=12 x-10 \\)",
        "confidence": 1,
        "is_printed": false,
        "request_id": "a4301400f66b9821d35cbabea8a26992",
        "is_handwritten": true,
        "confidence_rate": 1,
        "auto_rotate_degrees": 0,
        "auto_rotate_confidence": 0.0027393347044402105
      },
      "detections": {
        "contains_chemistry": false,
        "contains_diagram": false,
        "is_handwritten": true,
        "is_printed": false,
        "contains_table": false,
        "contains_triangle": false
      }
    },
    {
      "timestamp": "2021-07-02T18:48:45.903Z",
      "duration": 0.15,
      "endpoint": "/v3/text",
      "request_args": {
        "tags": ["test_tag", "test_tag2", "test_tag3"],
        "formats": ["text"]
      },
      "result": {
        "text": "\\( 12+5 x-8=12 x-10 \\)",
        "confidence": 1,
        "is_printed": false,
        "request_id": "7fe48fb1cd61f8a00ba4e71381740efb",
        "is_handwritten": true,
        "confidence_rate": 1,
        "auto_rotate_degrees": 0,
        "auto_rotate_confidence": 0.0027393347044402105
      },
      "detections": {
        "contains_chemistry": false,
        "contains_diagram": false,
        "is_handwritten": true,
        "is_printed": false,
        "contains_table": false,
        "contains_triangle": false
      }
    }
  ]
}

Search parameter	Type	Description
page (default=1)	number	First page of results to return
per_page (default=100)	number	Number of results to return
from_date (optional)	string	starting (included) ISO datetime
to_date (optional)	string	ending (excluded) ISO datetime
app_id (optional)	string	results for the given app_id
text (optional)	string	result.text contains the given string
text_display (optional)	string	result.text_display contains the given string
latex_styled (optional)	string	result.latex_styled contains the given string
tags (optional)	[string]	an array of tag strings
is_printed (optional)	boolean	All results that contain printed text/math are included
is_handwritten (optional)	boolean	All results that contain handwritten text/math are included
contains_table (optional)	boolean	All results that contain a table are included
contains_chemistry (optional)	boolean	All results that contain chemistry diagrams are included
contains_diagram (optional)	boolean	All results that contain diagrams are included
contains_triangle (optional)	boolean	All results that contain triangles are included

Result object

Field	Type	Description
timestamp	string	ISO timestamp of recorded result information
endpoint	object	API endpoint used for upload (eg `/v3/text`, `/v3/strokes`, ...)
duration	number	Difference between timestamp and when request was received
request_args	object	Request body arguments
result	object	Result body for request
detections	object	An object of detections for each request

Process a PDF

Mathpix supports PDF processing for scientific documents.

Supported outputs:

mmd file (Mathpix Markdown spec)
DOCX file (compatible with MS Office, Google Docs, Libre Office)
LaTeX zip file (includes images)

Request parameters

Send a PDF URL for processing:

#!/usr/bin/env python
import requests
import json

r = requests.post("https://api.mathpix.com/v3/pdf",
    json={"url": "http://cs229.stanford.edu/notes2020spring/cs229-notes1.pdf"},
    headers={
        "app_id": "APP_ID",
        "app_key": "APP_KEY",
        "Content-type": "application/json"
    }
)
print(json.dumps(r.json(), indent=4, sort_keys=True))

curl -X POST https://api.mathpix.com/v3/pdf \
-H 'app_id: APP_ID' \
-H 'app_key: APP_KEY' \
-H 'Content-Type: application/json' \
--data '{ "url": "http://cs229.stanford.edu/notes2020spring/cs229-notes1.pdf" }'

You can either send a PDF URL, or you can upload a file.

Parameter	Type	Description
url (optional)	string	HTTP URL where PDF can be downloaded from
metadata (optional)	object	Key value object
alphabets_allowed (optional)	object	See AlphabetsAllowed section, use this to specify which alphabets you don't want in the output
rm_spaces (optional)	bool	Determines whether extra white space is removed from equations in `latex_styled` and `text` formats. Default is `true`.
rm_fonts (optional)	bool	Determines whether font commands such as \mathbf and \mathrm are removed from equations in `latex_styled` and `text` formats. Default is `false`.
idiomatic_eqn_arrays (optional)	bool	Specifies whether to use aligned, gathered, or cases instead of an array environment for a list of equations. Default is `false`.
numbers_default_to_math (optional)	bool	Specifies whether numbers are always math, e.g., `Answer: $ 17 $` instead of `Answer: 17`. Default is `false`.
math_inline_delimiters (optional)	[string, string]	Specifies begin inline math and end inline math delimiters for `text` outputs. Default is `["\$", "\$"]`.
math_display_delimiters (optional)	[string, string]	Specifies begin display math and end display math delimiters for `text` outputs. Default is `["\\[", "\\]"]`.

Send a PDF file for processing:

import requests
import json

options = {
    "math_inline_delimiters": ["$", "$"],
    "rm_spaces": True
}
r = requests.post("https://api.mathpix.com/v3/pdf",
    headers={
        "app_id": "APP_ID",
        "app_key": "APP_KEY"
    },
    data={
        "options_json": json.dumps(options)
    },
    files={
        "file": open("css299-notes.pdf","rb")
    }
)
print(r.text.encode("utf8"))

curl --location --request POST 'https://api.mathpix.com/v3/pdf' \
--header 'app_id: APP_ID' \
--header 'app_key: APP_KEY' \
--form 'file=@"cs229-notes5.pdf"' \
--form 'options_json="{\"math_inline_delimiters\": [\"$\", \"$\"]}"'

To send a PDF file simply include the file in the form-data request body.

Response body

Reponse to PDF / PDF URL upload

{
  "pdf_id": "5049b56d6cf916e713be03206f306f1a"
}

Field	Type	Description
pdf_id	string	Tracking ID which will allow getting status and result, when finished
error (optional)	string	US locale error message
error_info (optional)	object	Error info object

Processing status

Response after a few seconds:

{
  "status": "split",
  "num_pages": 9,
  "percent_done": 11.11111111111111,
  "num_pages_completed": 1
}

Response after a few more seconds:

{
  "status": "completed",
  "num_pages": 9,
  "percent_done": 100,
  "num_pages_completed": 9
}

To check the processing status of a PDF, use the pdf_id returned from the initial request and append it to /v3/pdf in a GET request.

Field	Type	Description
status	string	Processing status, will be `received` upon successful request, `loaded` if PDF was down-loaded onto our servers, `split` when PDF pages are split and sent for processing,`complete` when PDF is done processing
num_pages (optional)	Integer	Total number of pages in PDF document
num_pages_completed (optional)	Integer	Current number of pages in PDF document that have been OCR-ed
percent_done (optional)	Float	Percentage of pages in PDF that have been OCR-ed

Result document

Save results to local mmd file, docx, file, and LaTeX zip

curl --location --request GET 'https://api.mathpix.com/v3/pdf/PDF_ID.mmd' \
--header 'app_key: APP_KEY' \
--header 'app_id: APP_ID' > PDF_ID.mmd

curl --location --request GET 'https://api.mathpix.com/v3/pdf/PDF_ID.docx' \
--header 'app_key: APP_KEY' \
--header 'app_id: APP_ID' > PDF_ID.docx

curl --location --request GET 'https://api.mathpix.com/v3/pdf/PDF_ID.tex' \
--header 'app_key: APP_KEY' \
--header 'app_id: APP_ID' > PDF_ID.tex.zip

import requests

pdf_id = "PDF_ID"
headers = {
  "app_key": "APP_KEY",
  "app_id": "APP_ID"
}

# get mmd response
url = "https://api.mathpix.com/v3/pdf/" + pdf_id + ".mmd"
response = requests.get(url, headers=headers)
with open(pdf_id + ".mmd", "w") as f:
    f.write(response.text)

# get docx response
url = "https://api.mathpix.com/v3/pdf/" + pdf_id + ".docx"
response = requests.get(url, headers=headers)
with open(pdf_id + ".docx", "wb") as f:
    f.write(response.content)

# get LaTeX zip file
url = "https://api.mathpix.com/v3/pdf/" + pdf_id + ".tex"
response = requests.get(url, headers=headers)
with open(pdf_id + ".tex.zip", "wb") as f:
    f.write(response.content)

Once a PDF has been fully OCR-ed, resulting in status=complete, you can get the actual OCR results by adding the desired extension to the status GET request.

The possible values of the extension are described here.

Extension	Description
mmd	Returns Mathpix Markdown text file
docx	Returns a docx file
tex	Returns a LaTeX zip file

Note that the tex extension actually downloads a zip file. This zip file contains the main .tex file, in addition to images that are referenced in the document.

Delete result document

To delete a PDF, use the DELETE method on the same URL as used for getting status or a document. When a PDF is deleted, the result MMD file and associated images get deleted from our servers. The PDF status is unaffected.

Deleting a PDF that hasn't completed processing will return a 404. Only PDFs that have status=complete can be deleted.

Query PDF results

Mathpix allows customers to search their results from posts to /v3/pdf with a GET request on /v3/pdf-results?search-parameters. The search request must contain a valid app_key header to identify the group owning the results to search.

Request parameters

curl -X GET -H 'app_key: APP_KEY' \
    'https://api.mathpix.com/v3/pdf-results?per_page=100&from_date=2020-06-26T03%3A08%3A22.827Z'

Query parameter	Type	Description
page (default=1)	number	First page of results to return
per_page (default=100)	number	Number of results to return
from_date (optional)	string	starting (included) ISO datetime
to_date (optional)	string	ending (excluded) ISO datetime
app_id (optional)	string	results for the given app_id

Query result object

{
  "pdfs": [
    {
      "id": "113cee229ab27b715b7cc24cfdbb2c33",
      "input_file": "https://s3.amazonaws.com/mathpix-ocr-examples/authors_0.pdf",
      "status": "completed",
      "created_at": "2020-06-26T03:08:23.827Z",
      "modified_at": "2020-06-26T03:08:27.827Z",
      "num_pages": 1,
      "num_pages_completed": 1,
      "request_args": {}
    },
    {
      "id": "9ad69e94346e65047215d25709760f29",
      "input_file": "https://s3.amazonaws.com/mathpix-ocr-examples/1457.pdf",
      "status": "completed",
      "created_at": "2020-06-27T03:08:23.827Z",
      "modified_at": "2020-06-27T03:08:27.827Z",
      "num_pages": 1,
      "num_pages_completed": 1,
      "request_args": {}
    }
  ]
}

Field	Type	Description
id	string	ID of the processed PDF
input_file	string	The URL or the filename of the processed PDF
created_at	string	ISO timestamp of when the PDF was received
modified_at	string	ISO timestamp of when the PDF finished processing
num_pages	number	Number of pages in the request PDF
num_pages_completed	number	Number of pages of the PDF that were processed
request_args	object	Request body arguments

Process a batch

An equation image batch request is made with JSON that looks like:

{
  "urls": {
    "inverted": "https://raw.githubusercontent.com/Mathpix/api-examples/master/images/inverted.jpg",
    "algebra": "https://raw.githubusercontent.com/Mathpix/api-examples/master/images/algebra.jpg"
  },
  "formats": ["latex_simplified"]
}

curl -X POST https://api.mathpix.com/v3/batch \
-H "app_id: APP_ID" \
-H "app_key: APP_KEY" \
-H "Content-Type: application/json" \
--data '{ "urls": {"inverted": "https://raw.githubusercontent.com/Mathpix/api-examples/master/images/inverted.jpg", "algebra": "https://raw.githubusercontent.com/Mathpix/api-examples/master/images/algebra.jpg"},"formats":["latex_simplified"] }'


import requests
import json

base_url = "https://raw.githubusercontent.com/Mathpix/api-examples/master/images/"

r = requests.post(
    "https://api.mathpix.com/v3/batch",
    json={
        "urls": {
            "algebra": base_url + "algebra.jpg",
            "inverted": base_url + "inverted.jpg"
        },
        "formats": ["latex_simplified"]
    },
    headers={
        "app_id": "APP_ID",
        "app_key": "APP_KEY",
        "content-type": "application/json"
    },
    timeout=30
)
reply = r.json()
assert "batch_id" in reply

The response to the batch is a positive integer value for batch_id.

{
  "batch_id": "17"
}

The response to GET /v3/batch/17 is below when the batch has completed. Before completion the "results" field may be empty or contain only one of the two results.

{
  "keys": ["algebra", "inverted"],
  "results": {
    "algebra": {
      "detection_list": [],
      "detection_map": {
        "contains_chart": 0,
        "contains_diagram": 0,
        "contains_geometry": 0,
        "contains_graph": 0,
        "contains_table": 0,
        "is_inverted": 0,
        "is_not_math": 0,
        "is_printed": 0
      },
      "latex_simplified": "12 + 5 x - 8 = 12 x - 10",
      "latex_confidence": 0.99640350138238,
      "position": {
        "height": 208,
        "top_left_x": 0,
        "top_left_y": 0,
        "width": 1380
      }
    },
    "inverted": {
      "detection_list": ["is_inverted", "is_printed"],
      "detection_map": {
        "contains_chart": 0,
        "contains_diagram": 0,
        "contains_geometry": 0,
        "contains_graph": 0,
        "contains_table": 0,
        "is_inverted": 1,
        "is_not_math": 0,
        "is_printed": 1
      },
      "latex_simplified": "x ^ { 2 } + y ^ { 2 } = 9",
      "latex_confidence": 0.99982263230866,
      "position": {
        "height": 170,
        "top_left_x": 48,
        "top_left_y": 85,
        "width": 544
      }
    }
  }
}

The Mathpix API supports processing multiple equation images in a single POST request to a different endpoint: /v3/batch. The request body may contain any /v3/latex parameters except src and must alsocontain a urls parameter. The request may also contain an additonal callback parameter to receive results after all the images in the batch have been processed.

Parameter	Type	Description
urls	object	key-value for each image in the batch where the value may be a string url or an object containing a url and image-specific request arguments such as `region` and `formats`.
callback (optional)	object	description of where to send the batch results
callback.post	string	url to post results
callback.reply (optional)	object	data to send in reply to batch POST
callback.body (optional)	object	data to send with results to callback.post
callback.headers (optional)	object	headers to use when posting results

The response contains only a unique batch_id value. Even if the request includes a callback, there is no guarantee the callback will run successfuly (because of a transient network failure, for example). The preferred approach is to wait an appropriate length of time (about one second for every five images in the batch) and then do a GET on /v3/batch/:id where :id is the batch_id value. The GET request must contain the same app_id and app_key headers as the POST to /v3/batch.

The GET response has the following fields:

Field	Type	Description
keys	string[]	all the url keys present in the originating batch request
results	object	an OCR result for each key that has been processed

Supported Image Types

Mathpix supports image types compatible with OpenCV, including:

JPEG files - _.jpeg, _.jpg, *.jpe
Portable Network Graphics - *.png
Windows bitmaps - _.bmp, _.dib
JPEG 2000 files - *.jp2
WebP - *.webp
Portable image format - _.pbm, _.pgm, _.ppm _.pxm, *.pnm
PFM files - *.pfm
Sun rasters - _.sr, _.ras
TIFF files - _.tiff, _.tif
OpenEXR Image files - *.exr
Radiance HDR - _.hdr, _.pic
Raster and Vector geospatial data supported by GDAL

Error handling

All requests will return error and error_info fields if an argument is missing or incorrect, or if some problem happens while processing the request. The error field contains an en-us string describing the error. The error_info field contains an object providing programmatic information.

Error info object

Field	Type	Description
id	string	specifies the error id (see below)
message	string	error message
detail (optional)	object	Additional error info

Error id strings

Id	Description	Detail fields	HTTP Status
http_unauthorized	Invalid credentials		401
http_max_requests	Too many requests	count	429
json_syntax	JSON syntax error		200
image_missing	Missing URL in request body		200
image_download_error	Error downloading image	url	200
image_decode_error	Cannot decode the image data		200
image_no_content	No content found in image		200
image_not_supported	Image is not math or text		200
image_max_size	Image is too large to process		200
strokes_missing	Missing strokes in request body		200
strokes_syntax_error	Incorrect JSON or strokes format		200
strokes_no_content	No content found in strokes		200
opts_bad_callback	Bad callback field(s)	post?, reply?, batch_id?	200
opts_unknown_ocr	Unknown ocr option(s)	ocr	200
opts_unknown_format	Unknown format option(s)	formats	200
opts_number_required	Option must be a number	name,value	200
opts_value_out_of_range	Value not in accepted range	name,value	200
pdf_encrypted	PDF is encrypted and not readable		200
pdf_unknown_id	PDF ID expired or isn't		200
pdf_missing	Request sent without url field		200
pdf_page_limit_exceeded	PDF exceeds maximum page limit		200
math_confidence	Low confidence		200
math_syntax	Unrecognized math		200
batch_unknown_id	Unknown batch id	batch_id	200
sys_exception	Server error		200
sys_request_too_large	Max request size is 5mb for images and 512kb for strokes		200

Privacy

Example of a request with the extra privacy setting:

{
  "metadata": {
    "improve_mathpix": false
  }
}

By default we make images accessible to our QA team so that we can make improvements.

We also provide an extra privacy option which ensures that no image data or derived information is ever persisted to disk, and no data is available to Mathpix's QA team (we still track the request and how long the request took to complete). Simply add a metadata object to the main request body with the improve_mathpix field set to false (by default it is true). Note that this option means that images and results will not be accessible via our dashboard (accounts.mathpix.com/ocr-api).

Long division

Response for image on the left side:

{
  "detection_map": {
    "contains_chart": 0,
    "contains_diagram": 0,
    "contains_geometry": 0,
    "contains_graph": 0,
    "contains_table": 0,
    "is_inverted": 0,
    "is_not_math": 0,
    "is_printed": 1
  },
  "latex_normal": "8 \\longdiv { 7200 }"
}

We use the special markup \longdiv to represent long division; it is the only nonvalid Latex markup we return. Long division is used much like \sqrt which is visually similar.

Latency considerations

The biggest source of latency is image uploads. The speed of a response from Mathpix API servers is roughly proportional to the size of the image. Try to use images under 100kb for maximum speeds. JPEG compression and image downsizing are recommended to ensure lowest possible latencies.

Vocabulary

Mathpix generates any of the following characters:

!	"	$	&	'
(	)	*	+	,
-	.	/	0	1
2	3	4	5	6
7	8	9	:	;
<	=	>	?	A
B	C	D	E	F
G	H	I	J	K
L	M	N	O	P
Q	R	S	T	U
V	W	X	Y	Z
[	\\	]	^	_
a	b	c	d	e
f	g	h	i	j
k	l	m	n	o
p	q	r	s	t
u	v	w	x	y
z	{	\|	}	~
\\%	\\\\	\\{	\\}
\alpha	\angle	\langle	\rangle	\approx
\because	\begin{array}	\beta	\bot	\cap
\cdot	\cdots	\chi	\circ	\cong
\cup	\dagger	\Delta	\delta	\div
\dot	\dots	\ell	\emptyset	\eta
\end{array}	\epsilon	\equiv	\exists	\kappa
\forall	\frac	\gamma	\Gamma	\geq
\hat	\hbar	\hline	\in	\infty
\int	\Lambda	\lambda	\lceil	\left(
\left.	\left[	\left\\{	\left\|	\Leftrightarrow
\leftrightarrow	\lfloor	\leq	\longdiv	\mathcal
\mp	\mu	\nabla	\neq	\notin
\oint	\Omega	\omega	\operatorna	\oplus
\overline	\otimes	\parallel	\partial	\perp
\Phi	\phi	\pi	\pm	\prime
\prod	\propto	\Psi	\psi	\qquad
\quad	\rceil	\rfloor	\rho	\right)
\right.	\right\}	\right]	\Rightarrow	\rightarrow
\right\|	\sigma	\sim	\simeq	\sqrt
\square	\star	\sum	\subset	\supset
\subseteq	\supseteq	\tau	\text	\therefore
\Theta	\theta	\times	\tilde
\varphi	\vdots	\vec	\wedge	\vee
\Xi	\xi	\zeta