NAV Navbar
Logo
shell python

Introduction

git clone git@github.com:Mathpix/api-examples.git
cd api-examples/python
sudo pip install requests
cat test.py

#!/usr/bin/env python
import sys
import base64
import requests
import json

file_path = sys.argv[1]
image_uri = "data:image/jpeg;base64," + base64.b64encode(open(file_path, "rb").read())
r = requests.post("https://api.mathpix.com/v3/latex",
    data=json.dumps({'url': image_uri}),
    headers={"app_id": "trial", "app_key": "34f1a4cea0eaca8540c95908b4dc84ab",
        "Content-type": "application/json"})
print json.dumps(json.loads(r.text), indent=4, sort_keys=True)
git clone git@github.com:Mathpix/api-examples.git
cd api-examples/python

Welcome to the Mathpix API! You send us an image, we tell you about the math that’s in it. It’s that simple. We return latex as well as image meta data that you can use in your application.

We have examples in Shell and Python! You can view code examples in the dark area to the right, and you can switch the programming language of the examples with the tabs in the top right.

To try the examples, please clone the examples repo from github and cd into the directory containing the examples, as shown on the right.

Processing a single image

Call the API as follows:

./test.py limit.jpg
curl -X POST https://api.mathpix.com/v3/latex \
    -H 'app_id: trial' \
    -H 'app_key: 34f1a4cea0eaca8540c95908b4dc84ab' \
    -H 'Content-Type: application/json' \
    --data '{ "url": "data:image/jpeg;base64,'$(base64 -i limit.jpg)'" }'

The API returns JSON structured as show here:

{
    "detection_list": [],
    "detection_map": {
        "contains_chart": 0,
        "contains_diagram": 0,
        "contains_geometry": 0,
        "contains_graph": 0,
        "contains_table": 0,
        "is_inverted": 0,
        "is_not_math": 0,
        "is_printed": 0
    },
    "error": "",
    "latex": "\\lim _ { x \\rightarrow 3} ( \\frac { x ^ { 2} + 9} { x - 3} )",
    "latex_confidence": 0.86757309488734,
    "position": {
        "height": 273,
        "top_left_x": 57,
        "top_left_y": 14,
        "width": 605
    }
}

Request parameters

The API expects a request must contain two headers (app_id and app_key) and a JSON body with a field named url that specifies an image. This field is a string either containing a public resource reference to use to retrieve the image content or a data url containing a base64 encoding of the image.

Parameter Description
app_id Application identifier (e.g. mymathsolver)
app_key Application key
content_type application/json
url string

Receiving results

The server responds with the results for the image unless the request contains a callback object, in which case the response only contains a single session_id field and the server asynchronously sends the result to the url specified by callback.post. The asynchronous post contains the result object and a reply object containing the session_id.

An application may override the session_id response by specifying a value for callback.reply in the request. The server will use this value both in the response and the asynchronous post.

Applications that wish to provide additional data in the post may specify a callback.body value. The server will pass this value to the post.

You can create a callback URL for testing simply by visiting https://requestb.in/. Sample code to test the async API option are on the right.

To test the async API, replace the callback URL below with your own by visiting https://requestb.in/.

#!/usr/bin/env python
import sys
import base64
import requests
import json

file_path = "integral.jpg"
image_uri = "data:image/jpeg;base64," + base64.b64encode(open(file_path, "rb").read())
r = requests.post("https://api.mathpix.com/v3/latex",
    data=json.dumps({'url': image_uri, 'callback': {'post': 'http://requestb.in/10z3g561', 'reply': file_path}}),
    headers={"app_id": "trial", "app_key": "34f1a4cea0eaca8540c95908b4dc84ab",
        "Content-type": "application/json"})
print r.text
curl -X POST https://api.mathpix.com/v3/latex \
    -H 'app_id: trial' \
    -H 'app_key: 34f1a4cea0eaca8540c95908b4dc84ab' \
    -H 'Content-Type: application/json' \
    --data '{ "url": "data:image/jpeg;base64,'$(base64 -i integral.jpg)'","callback":{"post": "http://requestb.in/10z3g561", "reply": "integral.jpg"}}'

The following JSON is delivered to the callback URL via a POST request:

{
  "reply": "integral.jpg",
  "result": {
    "detection_list": [
      "is_printed"
    ],
    "detection_map": {
      "contains_chart": 0,
      "contains_diagram": 0,
      "contains_geometry": 0,
      "contains_graph": 0,
      "contains_table": 0,
      "is_inverted": 0,
      "is_not_math": 0,
      "is_printed": 1
    },
    "error": "",
    "latex": "\\int \\frac { 4 x } { \\sqrt { x ^ { 2 } + 1 } } d x",
    "latex_confidence": 0.99817252453161,
    "latex_list": [],
    "position": {
      "height": 215,
      "top_left_x": 57,
      "top_left_y": 0,
      "width": 605
    }
  }
}

Results

Example output JSON for the following image is shown on the right.

Confidence values

The API returns confidence scores for the returned latex in the latex_confidence field. It is recommended that you discard results that are below a certain confidence value. You can tune this threshold value for your application. We use a 20% confidence cutoff for our iOS and Android demo apps.

Detection types

The API defines multiple detection types to the client inside the JSON field detection_map. The fields are described below:

Detection Definition
contains_chart Contains a visual representation of a discrete dataset.
contains_table Contains a tabular representation of a discrete dataset.
contains_diagram Contains a diagram.
contains_graph Contains a 1D, 2D, or 3D graph.
contains_geometry Contains chart, table, diagram, or graph.
is_printed The image is taken of printed math, not handwritten math.
is_not_math No valid equation was detected.

The API returns confidence values for each of these detection categories.

Detection coordinates

The position field contains the bounding box of the equation of interest, in pixel coordinates of the image sent. Note that we consider systems of algebraic equations and multiple line equations as single equations. If multiple equations are separated by a lot of text, we will return the first equation only.

Processing a batch of images

A batch request is made as follows:

curl -X POST https://api.mathpix.com/v3/batch \
    -H "app_id: trial" \
    -H "app_key: 34f1a4cea0eaca8540c95908b4dc84ab" \
    -H "Content-Type: application/json" \
    --data '{ "urls": {"inverted": "https://raw.githubusercontent.com/Mathpix/api-examples/master/python/inverted.jpg", "algebra": "https://raw.githubusercontent.com/Mathpix/api-examples/master/python/algebra.jpg"},"callback":{"post": "http://requestb.in/sr1x3lsr"}}'

import requests
import json

base_url = 'https://raw.githubusercontent.com/Mathpix/api-examples/master/python/'

data = {
    'urls': {
        'algebra': base_url + 'algebra.jpg',
        'inverted': base_url + 'inverted.jpg'
    },
    'callback': {
        'post': 'http://requestb.in/1ftatkr1'
    }
}

r = requests.post(
    "https://api.mathpix.com/v3/batch", data=json.dumps(data),
    headers={
        'app_id': 'trial',
        'app_key': '34f1a4cea0eaca8540c95908b4dc84ab',
        'content-type': 'application/json'
    },
    timeout=30
)
reply = json.loads(r.text)
assert reply.has_key('batch_id')

The callback response is as follows:

{
  "reply": {
    "batch_id": "11"
  }, 
  "result": {
    "algebra": {
      "detection_list": [], 
      "detection_map": {
        "contains_chart": 0, 
        "contains_diagram": 0, 
        "contains_geometry": 0, 
        "contains_graph": 0, 
        "contains_table": 0, 
        "is_inverted": 0, 
        "is_not_math": 0, 
        "is_printed": 0
      }, 
      "error": "", 
      "latex": "1 2 + 5 x - 8 = 1 2 x - 1 0 ", 
      "latex_confidence": 0.99640350138238, 
      "latex_list": [], 
      "position": {
        "height": 208, 
        "top_left_x": 0, 
        "top_left_y": 0, 
        "width": 1380
      }
    }, 
    "inverted": {
      "detection_list": [
        "is_inverted", 
        "is_printed"
      ], 
      "detection_map": {
        "contains_chart": 0, 
        "contains_diagram": 0, 
        "contains_geometry": 0, 
        "contains_graph": 0, 
        "contains_table": 0, 
        "is_inverted": 1, 
        "is_not_math": 0, 
        "is_printed": 1
      }, 
      "error": "", 
      "latex": "x ^ { 2 } + y ^ { 2 } = 9 ", 
      "latex_confidence": 0.99982263230866, 
      "latex_list": [], 
      "position": {
        "height": 170, 
        "top_left_x": 48, 
        "top_left_y": 85, 
        "width": 544
      }
    }
  }
}

The Mathpix API also supports processing multiple images in a single request to a different endpoint: /v3/batch. The request contains urls specifying a key-url pair for each image, and callback that specifies where to send the results. The response contains a unique batch_id value, and after processing all the images the server posts a message to callback.post containing result with a key-result pair for each image result and reply containing the batch_id.

An application may specify a value for callback.reply in the request to include fields in addition to batch_id in both the response and result post. However, the server-generated batch_id cannot be overridden.

Applications that wish to provide additional data in the post may specify a callback.body value. The server will pass this value to the post.

Latex conventions

We guarantee generating valid latex for every image. For simple equations that arise in arithmetic, algebra, trigonometry, and calculus, the latex we generate is exactly what you’d expect. However, due to the ambiguity of representing systems of equations and matrices in latex, we adopt certain conventions which we now describe.

Note that we ignore all trailing punctuation. We only return punctuation when it is used to separate elements on a single row.

Systems of equations

{
    "latex": "\\left\\{ \\begin{array} { l } { y = - \\frac { 5} { x } } \\\\ { y = - 5} \\end{array} \\right."
}

We use the array environment to represent systems of equations. The equations must be enclosed in \left[...] and \right[...] elements. For example, if a left brace is present but there is no right brace, we we will return a latex string as \left\{ ... \right.. If there are no braces on either side of the equation, we return \left. ... \right.. See examples on the right.

Matrices

{
    "latex": "\\left[ \\begin{array} { l l } { a } & { 5b } \\\\ { b } & { a } \\end{array} \\right]"
}

We also use array to represent matrices. See output on the right.

Long division

{
    "detection_map": {
        "contains_chart": 0,
        "contains_diagram": 0,
        "contains_geometry": 0,
        "contains_graph": 0,
        "contains_table": 0,
        "is_inverted": 0,
        "is_not_math": 0,
        "is_printed": 1
    },
    "latex": "8\\longdiv { 7200}"
}

The following <script> tags will take care of rendering the \longdiv markup.

<script type="text/javascript" src="//cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/x-mathjax-config">
    MathJax.Hub.Config({
        tex2jax: {inlineMath: [['$','$'], ['\\(','\\)']]},
        messageStyle: "none",
        TeX: {
            Macros: {
               longdiv: ["{\\overline{\\smash{)}#1}}", 1]
            }
        }
    });
</script>

We use the special markup \longdiv to represent long division. Long division is used much like \sqrt which is visually similar.

Problem numbers

{
    "latex": "y = \\frac { 2 x - 3 } { x + 1 } ",
    "latex_confidence": 0.99778992141393
}

We ignore problem numbers the precede the equation of interest (e.g. 1) or 1.).

Graphs

{
    "detection_list": [
        "contains_graph",
        "is_not_math",
        "contains_geometry",
        "is_printed"
    ],
    "detection_map": {
        "contains_chart": 0.0001,
        "contains_diagram": 0.0075,
        "contains_geometry": 1,
        "contains_graph": 0.9998,
        "contains_table": 0.0001,
        "is_inverted": 0,
        "is_not_math": 1,
        "is_printed": 0.9675
    },
    "error": "Sorry, can't figure this out! (equation not found)",
    "latex": "",
    "latex_confidence": 0,
    "latex_list": [],
    "position": {}
}

Geometry diagrams

{
    "detection_list": [
        "is_not_math",
        "contains_diagram",
        "contains_geometry",
        "is_printed"
    ],
    "detection_map": {
        "contains_chart": 0,
        "contains_diagram": 1,
        "contains_geometry": 1,
        "contains_graph": 0,
        "contains_table": 0,
        "is_inverted": 0,
        "is_not_math": 1,
        "is_printed": 1
    },
    "error": "Sorry, can't figure this out! (equation not found)",
    "latex": "",
    "latex_confidence": 0,
    "latex_list": [],
    "position": {}
}