Skip to main content

Process Stroke Data

Process handwriting stroke coordinates directly, without needing to generate an image first. Supports the same options as image processing but with lower latency.

Send stroke data​

Send stroke data to the v3/strokes endpoint. Strokes are represented as arrays of x and y coordinates, where each sub-array is one continuous stroke.

info

The request body has a double-nested strokes key:

  • The outer strokes is the API parameter name
  • The inner strokes contains the coordinate data (x and y arrays)

The example below sends four strokes that form the handwritten expression 3x23x^2:

Visualization of the four strokes in the example request. Each polyline is one continuous stroke rendered from the x and y coordinate arrays.
{
  "strokes": {
    "strokes": {
      "x": [
        [131,131,130,130,131,133,136,146,151,158,161,162,162,162,162,159,155,147,142,137,136,138,143,160,171,190,197,202,202,202,201,194,189,177,170,158,153,150,148],
        [231,231,233,235,239,248,252,260,264,273,277,280,282,283],
        [273,272,271,270,267,262,257,249,243,240,237,235,234,234,233,233],
        [296,296,297,299,300,301,301,302,303,304,305,306,306,305,304,298,294,286,283,281,281,282,284,284,285,287,290,293,294,299,301,308,309,314,315,316]
      ],
      "y": [
        [213,213,212,211,210,208,207,206,206,209,212,217,220,227,230,234,236,238,239,239,239,239,239,239,241,247,252,259,261,264,266,269,270,271,271,271,270,269,268],
        [231,231,232,235,238,246,249,257,261,267,270,272,273,274],
        [230,230,230,231,234,240,246,258,268,273,277,281,281,283,283,284],
        [192,192,191,189,188,187,187,187,188,188,190,193,195,198,200,205,208,213,215,215,215,214,214,214,214,216,218,220,221,223,223,223,223,221,221,220]
      ]
    }
  }
}
Example response
{
  "request_id": "cea6b8e4-0ab4-550a-c467-ce2eb00430be",
  "is_printed": false,
  "is_handwritten": true,
  "auto_rotate_confidence": 0.0020149118193977245,
  "auto_rotate_degrees": 0,
  "confidence": 1,
  "confidence_rate": 1,
  "latex_styled": "3 x^{2}",
  "text": "\\( 3 x^{2} \\)",
  "version": "SuperNet-200"
}

In the example response, the latex_styled field renders as:

3x23 x^{2}

Reading the response​

Each request is recognized on its own​

A request is recognized from the strokes it contains and nothing else. There is no memory of what you sent before, and sending the same session ID does not carry context forward.

This matters most for a live canvas, because the natural implementation — keep every stroke the user has drawn, and resend the whole set after each change — asks us what the whole page says. And we answer about the whole page:

One request holding two written lines
{
  "text": "\\( \\begin{array}{l}2 x+3=7 \\\\ 2 x=4\\end{array} \\)",
  "latex_styled": "\\begin{array}{l}\n2 x+3=7 \\\\\n2 x=4\n\\end{array}",
  "confidence": 1
}

Multiple expressions are combined into a single \begin{array}{l} block (or \begin{aligned} if you send idiomatic_eqn_arrays). This is not an error and not a setting you have missed — it is the correct answer to the question the request asked.

Send one expression per request. You get one clean LaTeX string back, each expression is recognized without the rest of the page influencing it, and the request stops growing as the page fills. Split the strokes on the client: you already know where the expressions are, because you captured the coordinates that drew them. A vertical-gap test between stroke bounding boxes, or a pen-idle timer, is enough — and both are instant, where asking us is a round trip.

latex_styled is optional — read text as well​

latex_styled is only present when the input can be rendered as a single LaTeX string. It is omitted when the input holds:

  • more than one line of text
  • a chemistry diagram
  • a chart
  • a table with merged cells (\multirow / \multicol)

text is populated whenever anything is recognized, as Mathpix Markdown with math in \( \) and \[ \] delimiters. So a client that renders only when latex_styled is present will silently do nothing on those responses — and on a live canvas that reads as the display freezing on whatever it last drew, rather than as an error.

// Wrong: does nothing at all on any response without latex_styled.
if (result.latex_styled) render(result.latex_styled);

// Right: latex_styled when it is there, text otherwise.
const math = result.latex_styled || result.text;
if (math) render(math);

Live stroke sessions​

For live digital ink with updating results (e.g., in a mobile app), use app tokens with stroke sessions:

  1. Get an app_token with strokes_session_id from your server:
POST /v3/app-tokens
{
  "include_strokes_session_id": true,
  "expires": 300
}
  1. Use the app_token and strokes_session_id in client requests to v3/strokes.

Live stroke sessions are billed differently from standalone requests — see pricing. You are billed the first time strokes are sent for a session, not when requesting the token.

Working example​

Mathpix/live-math-drawing-demo is a small open-source React app that implements this whole loop: a drawing canvas, a live stroke session, and the recognized math rendered underneath as you write.

Three things it already solves that are easy to get wrong:

  • Rolling session renewal. A stroke session token lives at most 300 seconds. The demo mints a replacement shortly before the current one expires and carries on without interrupting the drawing, which is what any session longer than five minutes needs.
  • Strikethrough and scribble to delete. Draw a line through something already written and those strokes are removed from the set before the next request.
  • The full render path, from stroke coordinates to displayed math.

Note that the demo requests the app token from the browser, so the API key ends up in the client bundle. That is fine for running it locally, but in production request the token from your own server and hand only the token to the client — see app tokens.

Capturing good strokes​

Recognition quality depends heavily on how many points your client actually captures. A canvas driven by mousemove receives at most one point per rendered frame, roughly a quarter of what a stylus reports, and short fast strokes can degenerate to a single point.

  • Listen for pointer events (pointerdown / pointermove / pointerup / pointercancel) rather than mouse events, and set touch-action: none on the canvas.
  • Inside pointermove, record every point from event.getCoalescedEvents(), not just the event's own coordinate. The browser buffers the samples the digitizer produced between frames and discards them unless you ask.
  • Call canvas.setPointerCapture(event.pointerId) on pointerdown, so a stroke that leaves the canvas is not silently truncated.
  • Record the pointerdown and pointerup coordinates as points too.
  • Size the canvas for devicePixelRatio, or coordinates are quantized to CSS pixels.

Send the raw sample points. Do not smooth or thin them first — smoothing is for what you draw on screen, not for what you send.

Next steps​