<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title><![CDATA[Madhu’s Dev Notes]]></title><description><![CDATA[Madhu’s Dev Notes]]></description><link>https://madhushankarox.hashnode.dev</link><generator>RSS for Node</generator><lastBuildDate>Thu, 12 Nov 2026 09:29:52 GMT</lastBuildDate><atom:link href="https://madhushankarox.hashnode.dev/rss.xml" rel="self" type="application/rss+xml"/><language><![CDATA[en]]></language><ttl>60</ttl><item><title><![CDATA[Webflow only pastes its own elements. Here’s how I got it to take SVG and HTML.]]></title><description><![CDATA[How Webflow’s clipboard format works, and how I turned SVG icons and HTML with CSS into native Webflow elements, classes and breakpoints.]]></description><link>https://madhushankarox.hashnode.dev/paste-svg-and-html-into-webflow</link><guid isPermaLink="true">https://madhushankarox.hashnode.dev/paste-svg-and-html-into-webflow</guid><category><![CDATA[webflow]]></category><category><![CDATA[webflow design]]></category><category><![CDATA[JavaScript]]></category><dc:creator><![CDATA[Madhushanka]]></dc:creator><pubDate>Sun, 08 Nov 2026 16:44:41 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/covers/6aef2012f07c51fd48609787/291ec457-43de-458a-bf35-afd30d59d7a5.jpg" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p><em>Copy an icon from Figma, press Ctrl+V in the Webflow Designer, and nothing happens. The Designer only accepts its own elements. This is how its clipboard works, and how I convert SVG and HTML into it.</em></p>
<p>I run <a href="https://devpack.app/">devpack</a>, a site of free browser tools, and a lot of them are for Webflow. Webflow’s Designer is great when everything starts inside Webflow. It gets awkward when it doesn’t: an icon from Figma, a section someone wrote in plain HTML and CSS, a block of code ChatGPT gave you.</p>
<p>You copy it, click into the Designer, press Ctrl+V, and nothing happens. The Designer only pastes its own elements.</p>
<p>The usual workaround is a Code Embed. Paste the SVG or HTML into an Embed element and it shows up on the page. But an Embed is a black box. You can’t select the path inside it, you can’t give it a class, and the styles don’t show up in the Style panel. With twenty icons, you have twenty little boxes of code to keep in sync.</p>
<p>So I built two tools, <a href="https://devpack.app/svg-to-webflow">SVG to Webflow</a> and <a href="https://devpack.app/html-css-to-webflow">HTML &amp; CSS to Webflow</a>, and then a browser extension that does the same thing right inside the Designer when you paste. This is how they work.</p>
<h2>What Webflow puts on the clipboard</h2>
<p>The first step was to find out what Webflow pastes. When you copy an element in the Designer, it doesn’t put HTML on the clipboard. It puts JSON there, under the <code>application/json</code> type (and the same text as <code>text/plain</code>). You can see it by pasting into any text editor.</p>
<p>Trimmed down, a copied div with a class looks like this:</p>
<pre><code class="language-json">{
  "type": "@webflow/XscpData",
  "payload": {
    "nodes": [
      { "_id": "4f1c…", "type": "Block", "tag": "div",
        "classes": ["9a2e…"], "children": [], "data": { … } }
    ],
    "styles": [
      { "_id": "9a2e…", "type": "class", "name": "card", "comb": "",
        "styleLess": "padding: 24px; border-radius: 12px;",
        "variants": { "small": { "styleLess": "padding: 16px;" } },
        "children": [] }
    ],
    "assets": [], "ix1": [], "ix2": { … }
  },
  "meta": { … }
}
</code></pre>
<p>A few things stood out:</p>
<ul>
<li>Elements are a flat list of <code>nodes</code>. Each one has an id, and parents list their children by id. Text is its own node too, with <code>text: true</code>.</li>
<li>Classes live in a separate <code>styles</code> list, and the CSS is a plain string called <code>styleLess</code>.</li>
<li>Breakpoints and states are <code>variants</code>, keyed by names like <code>medium</code>, <code>small</code>, <code>tiny</code> and <code>main_hover</code>. If I could build this JSON myself and put it on the clipboard, the Designer would paste it like anything you copied in Webflow.</li>
</ul>
<h2>Putting JSON on the clipboard from a web page</h2>
<p>This sounded like the easy part. It wasn’t.</p>
<p>The modern Clipboard API, <code>navigator.clipboard.write()</code>, only lets a page write a short list of types: plain text, HTML and PNG images. Chrome also allows custom types, but only with a <code>"web "</code> prefix, and Webflow doesn’t read those. There’s no way to write a plain <code>application/json</code> entry with it.</p>
<p>The old way still works, though. If you handle the <code>copy</code> event yourself, you can set any type you like on the event’s <code>clipboardData</code>. Then you trigger a copy from a button click:</p>
<pre><code class="language-js">function copyAs(map) {
  let ok = false;
  const onCopy = e =&gt; {
    for (const [type, value] of Object.entries(map)) e.clipboardData.setData(type, value);
    e.preventDefault();
    ok = true;
  };
  document.addEventListener('copy', onCopy);
  try { document.execCommand('copy'); }
  finally { document.removeEventListener('copy', onCopy); }
  return ok;
}
 
copyAs({ 'application/json': json, 'text/plain': json });
</code></pre>
<p><code>execCommand</code> is deprecated, but every browser still supports it for this, and for now it’s the only way to do it.</p>
<h2>Turning an SVG into elements</h2>
<p>Webflow has a Custom Element: an element where you choose the tag yourself. That’s what makes SVG possible. An <code>&lt;svg&gt;</code> becomes a Custom Element with the tag <code>svg</code>, every <code>&lt;path&gt;</code> becomes a Custom Element with the tag <code>path</code>, and all the attributes (<code>viewBox</code>, <code>d</code>, <code>fill</code>…) go into the element’s attribute list. In the Navigator you get an svg element with its paths inside, and you can select and style each one.</p>
<img src="https://devpack.app/assets/img/blog/webflow-paste-svg-tool.jpg" alt="The SVG to Webflow tool: SVG code exported from Figma on the left, and a preview of the icon with the cleaned SVG on the right" style="display:block;margin:0 auto" />

<p>Before converting, the SVG needs cleaning. What you copy out of Figma or an icon site carries a lot of extras:</p>
<ul>
<li>Editor attributes like <code>data-name</code>, <code>sodipodi:*</code>, <code>inkscape:*</code> and <code>figma:*</code>, plus comments, <code>&lt;metadata&gt;</code> and <code>&lt;title&gt;</code>. They go.</li>
<li>Anything that can run code: <code>&lt;script&gt;</code>, <code>&lt;foreignObject&gt;</code>, <code>on*</code> attributes and <code>javascript:</code> links. They go too.</li>
<li><code>xlink:href</code>. This one took me a while. The Designer recreates the elements without the xlink namespace, so a <code>&lt;use xlink:href="#icon"&gt;</code> silently shows nothing. Rewriting it as a plain <code>href</code> fixes it. There are two options I added because I kept wanting them myself. <strong>currentColor</strong> replaces every fill and stroke color with <code>currentColor</code>, so the icon follows the text color you set in Webflow. And a <strong>class name and size</strong> option puts the width and height on a Webflow class instead of the SVG, so all your icons share one class you can change in one place.</li>
</ul>
<h2>The format isn’t documented, and it changes</h2>
<p>None of this JSON is documented, and Webflow can change the exact shape of a Custom Element whenever it likes. A converter that hard-codes it breaks quietly one day.</p>
<p>So the tools can learn it. If you copy any Custom Element in the Designer and paste it into the tool (or, with the extension, paste it back into the Designer), it reads the element and uses it as a template. The tricky part is finding where the attributes live, because that key could move. I search for it instead of hard-coding it: the first array whose items all look like <code>{ name, value }</code>.</p>
<pre><code class="language-js">function findAttrPath(obj, path = []) {
  for (const [k, v] of Object.entries(obj)) {
    if (Array.isArray(v) &amp;&amp; v.length &amp;&amp; v.every(x =&gt; x &amp;&amp; 'name' in x &amp;&amp; 'value' in x)) return [...path, k];
    if (v &amp;&amp; typeof v === 'object' &amp;&amp; !Array.isArray(v)) {
      const found = findAttrPath(v, [...path, k]);
      if (found) return found;
    }
  }
  return null;
}
</code></pre>
<p>Every new element is a copy of that template with a fresh id, its own tag and its own attributes. If Webflow changes the format, one copy and paste updates it.</p>
<h2>HTML and CSS: classes, breakpoints and hover states</h2>
<p>SVG was the warm-up. HTML with CSS is where it gets interesting, because the goal isn’t just elements. It’s elements with real Webflow classes, so you can keep editing them in the Style panel.</p>
<p>The elements part is a walk over the parsed HTML. In “native” mode, tags map to Webflow’s own elements: <code>div</code>, <code>section</code>, <code>header</code> and the other layout tags become Div Blocks that keep their tag, <code>h1</code>–<code>h6</code> become Headings, <code>p</code> a Paragraph, <code>a</code> a Link, <code>img</code> an Image, and lists become List and List Item. Anything else, including inline SVG, becomes a Custom Element with its exact tag. There’s also a “custom” mode that keeps every tag exactly as written.</p>
<p>The CSS is the hard part, because Webflow and CSS think about styles differently:</p>
<ul>
<li><strong>CSS is a cascade of rules. Webflow has one style per class.</strong> A class can be set by several rules in your CSS, and the order and specificity decide which values win.</li>
<li><strong>Your media queries can be anything. Webflow has fixed breakpoints, and they’re desktop-first.</strong> The base style is desktop, and each smaller breakpoint (991px, 767px and 479px wide) only stores what changes.</li>
<li><strong>Combo classes.</strong> <code>.button.is-secondary</code> in CSS is a combo class in Webflow, which is a separate style that belongs to its parent class. So instead of translating rules one by one, I work out what each class actually looks like at each breakpoint. For every class, I pick one test width per breakpoint (1200, 900, 600 and 400 pixels), collect all the rules that apply at that width, sort them by specificity and order, and get the final values. That gives me the full desktop style. For each smaller breakpoint, I store only the values that differ from the one above it:</li>
</ul>
<pre><code class="language-js">const BANDS = [{ key: 'main', w: 1200 }, { key: 'medium', w: 900 },
               { key: 'small', w: 600 }, { key: 'tiny', w: 400 }];
 
const effs = BANDS.map(b =&gt; effectiveStyle(cls, b.w));   // final values at each width
const variants = {};
BANDS.forEach((b, i) =&gt; {
  if (!i) return;
  const changed = diff(effs[i], effs[i - 1]);              // only what changes
  if (Object.keys(changed).length) variants[b.key] = { styleLess: toLess(changed) };
});
</code></pre>
<p>This handles <code>max-width</code> and <code>min-width</code> queries, rems and ems, and queries written in any order, because I never try to map a query to a breakpoint directly. I just test each width. <code>:hover</code>, <code>:focus</code> and <code>:active</code> work the same way and become Webflow states like <code>main_hover</code>. Shorthands get expanded too, since Webflow stores longhands: <code>padding: 8px 16px</code> becomes four separate values. <code>var(--…)</code> values are resolved from <code>:root</code>.</p>
<p>Some CSS just can’t become a Webflow class: element selectors like <code>h1 { … }</code>, descendant selectors, <code>@keyframes</code> and media queries for screens wider than Webflow’s base breakpoint. I don’t want to throw that away silently, so it goes into a Code Embed called “Custom CSS” that’s pasted along with the elements. The tool also shows a report: how many elements, classes and combo classes it created, and a warning for every rule it couldn’t map, every image with a relative URL (Webflow needs absolute ones) and an Embed that’s over Webflow’s 50,000-character limit.</p>
<img src="https://devpack.app/assets/img/blog/webflow-paste-report.jpg" alt="The HTML and CSS to Webflow tool after converting a hero section: a preview of the section, then 12 elements, 11 classes, 1 combo class, 4 breakpoint and hover styles, and a report that says everything mapped to native Webflow classes" style="display:block;margin:0 auto" />

<h2>The extension: catching the paste</h2>
<p>The web tools have one annoying step: you paste into the tool, click Copy, switch tabs, then paste into Webflow. I wanted Ctrl+V in the Designer to just work. That needs a browser extension, because only an extension can run code on the Designer’s page.</p>
<p>The extension’s content script listens for <code>paste</code> in the capture phase, before the Designer sees it, and decides what’s on the clipboard:</p>
<ul>
<li>Only an SVG? Convert it to elements.</li>
<li>Text that looks like HTML code? Convert it with the CSS from its <code>&lt;style&gt;</code> tags.</li>
<li>Rich HTML from a web page, like part of an article? Convert it to clean Headings, Paragraphs, Lists, Links and Images, without the page’s classes and styles.</li>
<li>Anything else is left alone: real Webflow data, plain text, pastes into text fields and code editors, and rich copies from VS Code or GitHub, which are code, not content. Then it stops the original paste and fires a new one, carrying the converted JSON:</li>
</ul>
<pre><code class="language-js">window.addEventListener('paste', e =&gt; {
  if (!e.isTrusted) return;            // our own paste passes straight through
  const what = classify(e.clipboardData);
  if (!what) return;
  e.preventDefault();
  e.stopImmediatePropagation();
 
  const json = convert(what);
  const data = new DataTransfer();
  data.setData('application/json', json);
  data.setData('text/plain', json);
  e.target.dispatchEvent(new ClipboardEvent('paste', { clipboardData: data, bubbles: true, cancelable: true }));
}, true);
</code></pre>
<p>The <code>isTrusted</code> check matters. The event I dispatch is untrusted, so the listener ignores it and doesn’t loop forever, and the Designer handles it like a normal paste.</p>
<p>Firefox needed a different route. There, a page can’t read the data of a paste event that an extension created, so the hand-off arrives empty. In Firefox the extension puts the converted JSON on the real clipboard instead, using the same copy-event trick as the website, and shows a message: press Ctrl+V again. The first paste converts, the second one pastes.</p>
<p>It also keeps a small component library. When you copy elements in the Designer, it offers to save them, and Alt+Shift+L opens your saved components in any project. I won’t go into that here, it’s a post of its own.</p>
<p>The extension asks for one permission, <code>storage</code>, for your settings and library. It only runs on Webflow Designer pages and devpack.app, and it makes no network requests. Everything is converted on your machine.</p>
<h2>One more catch: two browsers</h2>
<p>Custom clipboard types don’t travel between browsers. If you copy in Chrome and Webflow is open in Firefox, or the other way around, the JSON arrives as plain text and the Designer ignores it.</p>
<p>For that I made a small page, the <a href="https://devpack.app/webflow-clipboard-bridge">Webflow Clipboard Bridge</a>. Open it in the browser where Webflow is, paste there, and it copies the data again in that browser’s own format.</p>
<h2>What doesn’t work (yet)</h2>
<ul>
<li>Element selectors, descendant selectors and animations end up in the Custom CSS embed, not in Webflow classes. Inline <code>style</code> attributes are dropped with a warning, so move them into a class first.</li>
<li>Images need to be uploaded to Webflow afterwards. The converter keeps absolute image URLs and warns about relative ones.</li>
<li>Webflow can change its clipboard format. The template learning covers Custom Elements, but a big change would still need an update from me.</li>
</ul>
<h2>If you’re building something similar</h2>
<ul>
<li><strong>Look at the clipboard first.</strong> Paste what an app copies into a text editor. Many apps use plain JSON you can generate yourself.</li>
<li><strong>To write a custom type from a page, handle the</strong> <code>copy</code> <strong>event.</strong> <code>navigator.clipboard.write()</code> won’t do it.</li>
<li><strong>Don’t translate CSS rule by rule.</strong> Work out the final values at each breakpoint and store only the differences.</li>
<li><strong>Don’t hard-code an undocumented format.</strong> Learn it from a real sample, and search for the parts you need.</li>
<li><strong>In an extension, re-dispatch the paste and check</strong> <code>isTrusted</code><strong>.</strong> And test Firefox early, because it handles synthetic clipboard events differently. The tools are free: <a href="https://devpack.app/svg-to-webflow">SVG to Webflow</a> and <a href="https://devpack.app/html-css-to-webflow">HTML &amp; CSS to Webflow</a>. The extension is free too, for <a href="https://chromewebstore.google.com/detail/oghihmhkdhfpniljdfjfngapkibkbmkn">Chrome, Edge and other Chromium browsers</a> and for <a href="https://addons.mozilla.org/firefox/addon/paste-svg-html-into-webflow/">Firefox</a>. If you have an SVG or a bit of HTML that doesn’t convert well, I’d like to see it.</li>
</ul>
<hr />
<p><em>Originally published on the</em> <a href="https://devpack.app/blog/paste-svg-and-html-into-webflow"><em>devpack blog</em></a><em>.</em></p>
]]></content:encoded></item><item><title><![CDATA[My browser OCR couldn't read a sunscreen bottle. Here's what fixed it.]]></title><description><![CDATA[Why Tesseract.js turned phone photos into gibberish, and the fixes that worked: EXIF rotation, inverted text and merging OCR passes.]]></description><link>https://madhushankarox.hashnode.dev/making-browser-ocr-read-phone-photos</link><guid isPermaLink="true">https://madhushankarox.hashnode.dev/making-browser-ocr-read-phone-photos</guid><category><![CDATA[OCR ]]></category><category><![CDATA[JavaScript]]></category><category><![CDATA[Web Development]]></category><category><![CDATA[image processing]]></category><dc:creator><![CDATA[Madhushanka]]></dc:creator><pubDate>Sat, 07 Nov 2026 13:08:57 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/covers/6aef2012f07c51fd48609787/e8f8716f-4ff9-4237-bcca-15d76a18f9bb.jpg" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Screenshots worked fine. Then someone pointed a phone at a bottle of sunscreen, and my image-to-text tool spat out “EB ewraone”. This is what was going wrong and what I changed.</p>
<p>I run <a href="https://devpack.app/">devpack</a>, a site with 500+ free tools that work in the browser. One of them is an <a href="https://devpack.app/image-to-text">image to text</a> tool. It uses Tesseract.js, which is the Tesseract OCR engine compiled to WebAssembly, so the image never leaves your device. I liked that a lot, and for screenshots and scanned pages it worked really well. Most of my tests came back at 94 to 96% confidence.</p>
<p>Then I tried it the way people actually use their phones. I took a photo of a kids’ sunscreen bottle. Blue plastic, white letters, a big yellow sun with “SPF 50” in it. This is what came back:</p>
<pre><code class="language-plaintext">| FA Be NO
k Jody ; "
| Diana
i Collection
Kid i
“protect &amp; care
v4 oR. a
7 SPF
...
EB ewraone
</code></pre>
<p>A few real words in there, buried in junk. A second photo, a baby cologne bottle, was worse. It came back as seven short lines. One of them was just an ampersand, and another was “&amp; TED”.</p>
<p>It turned out there wasn’t one problem. There were four, and they stacked on top of each other.</p>
<h2>Problem 1: the engine was reading the photo sideways</h2>
<p>This one took me longer to see than I’d like to admit, because on my screen the photo looked perfectly fine.</p>
<p>Phone cameras often don’t rotate the actual pixels when you hold the phone upright. They save the image the way the sensor sees it and add a small piece of metadata, the EXIF orientation tag, that says “turn this 90° when you show it”. Your gallery app obeys the tag. Browsers obey it too when they display an <code>&lt;img&gt;</code>. But I was handing the raw <code>File</code> straight to Tesseract, and Tesseract just reads the pixels.</p>
<img src="https://devpack.app/assets/img/blog/ocr-sideways.jpg" alt="The same cologne photo twice: on the left lying sideways, the way the pixels are stored, and on the right upright, the way the phone shows it" style="display:block;margin:0 auto" />

<p>So the engine was trying to read text that was lying on its side. No wonder it found “ING” and an ampersand.</p>
<p>The fix is small. Decode the file with <code>createImageBitmap</code> and ask for the orientation to be applied, then draw it on a canvas and use that canvas for everything afterwards:</p>
<pre><code class="language-plaintext">async function uprightCanvas(file) {
  const bmp = await createImageBitmap(file, { imageOrientation: 'from-image' });
  const c = document.createElement('canvas');
  c.width = bmp.width;
  c.height = bmp.height;
  c.getContext('2d').drawImage(bmp, 0, 0);
  bmp.close();
  return c;
}
</code></pre>
<p>I also use this same canvas for the preview thumbnail now. Before, the tool showed you nothing at all, so you couldn’t even tell what it was looking at. Now you see exactly what the OCR sees, and if it’s wrong there’s a rotate button.</p>
<p>One more case: some photos really are sideways, with no tag at all. Screenshots of a rotated photo, or images that went through an app that stripped the metadata. For those, if the first read comes back with low confidence, the tool reads a small copy (about 1000 px) at 0°, 90°, 180° and 270°, scores each one, and turns the image if one direction is clearly better. The build of Tesseract.js I use only has the LSTM engine, so its own orientation detection isn’t available. Four quick passes on a small copy did the job instead.</p>
<h2>Problem 2: white text on a colored background</h2>
<p>With the rotation fixed, the cologne bottle got better, but the sunscreen was still rough. Look at the label again. Most of the important words, like “UVA / UVB PROTECTION” and “NET WT 200 ml”, are white or light blue on dark blue.</p>
<p>Tesseract is trained mostly on dark text on light paper. It handles some inverted text, but on a photo with mixed colors, reflections and a curved surface, it mostly gives up.</p>
<p>So I make two extra copies of the image. The first one is plain grayscale with a contrast stretch: I build a histogram, throw away the darkest and brightest 1% of pixels, and spread the rest over the full 0 to 255 range. The second copy is the same thing, inverted. White-on-blue becomes black-on-white.</p>
<img src="https://devpack.app/assets/img/blog/ocr-invert.jpg" alt="Part of the sunscreen label: on the left white UVA / UVB PROTECTION text on blue, on the right the same area as black text on a light gray background" style="display:block;margin:0 auto" />

<p>The code is not fancy:</p>
<pre><code class="language-plaintext">// a: ImageData pixels, already grayscale; lo/hi: the 1% and 99% points
for (let i = 0; i &lt; a.length; i += 4) {
  let g = (a[i] - lo) * 255 / (hi - lo);
  g = Math.max(0, Math.min(255, g));
  if (invert) g = 255 - g;
  a[i] = a[i + 1] = a[i + 2] = g;
}
</code></pre>
<p>I also switch Tesseract to page segmentation mode 11 (“sparse text”) for these passes. The default mode 3 expects a page with paragraphs and columns. A product label is more like a handful of words scattered around a picture, and mode 11 is built for exactly that.</p>
<h2>Problem 3: no single pass gets everything</h2>
<p>This was the part I didn’t expect. I assumed one of the variants would just win. It doesn’t work like that.</p>
<p>On the sunscreen bottle, the normal pass found “Diana” and “Collection”. The inverted pass found “UVA UVB PROTECTION” and “NET WT 200 ml”. The original color image, read as a document, was the only one that caught “Water Resistant” on one of my test runs. Every pass also added its own garbage: “Ag”, “as”, “©DDL”, “CJC)”. That’s the sun and the cartoon kid getting read as letters.</p>
<p>So instead of picking one result, I merge them word by word. Tesseract gives you every word with a bounding box and a confidence score, which makes this possible:</p>
<ol>
<li>Collect the words from all passes.</li>
<li>Throw out the obvious junk. Words of three or more characters need at least 60% confidence. One and two character fragments need 88%, because that’s where most of the noise lives. Anything with stray symbols like © or an unmatched bracket goes too.</li>
<li>If two passes read the same word in the same spot, give it a confidence boost. Two readings agreeing is a much better signal than one reading being sure of itself. This is what saved “Slumber” on the cologne bottle, which came in at 79% in one pass and 54% in another.</li>
<li>Where boxes overlap, keep the better word. At first I picked the most confident one, and the result was “Pea” instead of “Peaceful”, because the short fragment happened to score higher. Now the score favors whole words a bit: confidence times the square root of the length.</li>
<li>Rebuild the lines by grouping words whose vertical centers are close, then sort each line left to right.</li>
</ol>
<p>Small print got its own fix: on images that aren’t already large, I run the grayscale and inverted passes again on a copy scaled up 1.6×. That’s where “MOISTURISING” and “ALCOHOL FREE” on the cologne bottle finally showed up.</p>
<h2>What it reads now</h2>
<p>Here’s the sunscreen bottle after all of this, at 92% average confidence:</p>
<pre><code class="language-plaintext">Diana
Collection
Kid
protect &amp; care
UVA UVB
PROTECTION
face &amp; body
sunblock lotion
NET WT 200 ml
</code></pre>
<p>Compare that with the first output. Every line of plain printed text on that label is there.</p>
<p>The cologne bottle is the one on the cover. I retook it on a table, in plain light, and ran both versions on the same photo. The old single pass got 37% confidence and gave me “QUPPAGES”, “on) 9 fed” and “J NL Sil”. The new version gets 90%:</p>
<pre><code class="language-plaintext">velOna
UDDLES
MOISTURISING
COLOGNE
ALCOHOL FREE
Peaceful Slumber
DERMATOLOGICALLY
TESTED
Nn nie
</code></pre>
<p>Not perfect. “Nn nie” is the little drop icon and the edge of the label, and the bubble letters still lose their first C. But every line you’d actually want to copy is there.</p>
<p>The important thing for me was not breaking what already worked. Screenshots and scans go through the first, normal pass and come back with high confidence, so the extra passes never run. A receipt still reads at 94% and a notes screenshot at 95%, at the same speed as before. The photo mode only kicks in when that first pass is unsure, and it adds about two or three seconds on my laptop.</p>
<h2>Bonus: guessing the language</h2>
<p>Tesseract needs to know the language up front, and most people never touch the dropdown. So the tool now has an “Auto-detect” option, and it’s the default on the English site.</p>
<p>The trick is simple. It reads the image in English first. English models still read Spanish, French or German letters reasonably well, just without the accents. Then I strip the accents from the result and count common short words for each language: “the”, “and” and “of” for English, “el”, “la” and “que” for Spanish, “der”, “die” and “und” for German, and so on. If another language clearly wins, the tool downloads that language’s data and reads the image again. A Spanish test note came back at 96% with “reunión” and “mañana” spelled correctly.</p>
<p>This doesn’t work for other scripts. You can’t tell Russian from Hindi by reading them with an English model, because you just get nonsense. For those, the tool looks at the languages set in your browser, and if one of them uses another script and the English result is weak, it tries that language.</p>
<h2>What still doesn’t work</h2>
<p>I want to be honest about this part. The cologne bottle says “CUDDLES” in big bubble letters, and the best the tool manages is “UDDLES”. The “Lady” in script on the sunscreen doesn’t come through at all, and neither does handwriting. These are limits of the engine, and the fix would be a much bigger model. That means tens of megabytes to download, or sending the photo to a server, and the whole point of this tool is that your images stay on your device.</p>
<p>For printed text on real-world photos, though, it went from mostly unusable to pretty good.</p>
<h2>If you’re building something similar</h2>
<ul>
<li><strong>Always apply EXIF orientation before you process a photo.</strong> <code>createImageBitmap(file, { imageOrientation: 'from-image' })</code> does it in one line.</li>
<li><strong>Show the user the image you’re actually processing.</strong> A preview would have shown me the sideways problem in five seconds.</li>
<li><strong>Read light-on-dark text from an inverted copy.</strong> Don’t trust the engine to handle it.</li>
<li><strong>Merge several passes instead of choosing one.</strong> Word boxes and confidence scores make it straightforward, and agreement between passes is a great signal.</li>
<li><strong>Only pay for the extra work when you need it.</strong> A confident first pass means you can skip everything else.</li>
</ul>
<p>If you want to try it, the <a href="https://devpack.app/image-to-text">Image to Text</a> tool is free and runs entirely in your browser. On a phone there’s a “Take a photo” button. I’d genuinely like to see the photos it still gets wrong.</p>
<hr />
<p><em>Originally published on the</em> <a href="https://devpack.app/blog/making-browser-ocr-read-phone-photos"><em>devpack blog</em></a><em>.</em></p>
]]></content:encoded></item></channel></rss>