Text parsing · · updated
RegEx engine optimization for in-browser log filtering
In-browser filtering is a loop of predicates over strings already in RAM. JavaScript’s regex engine backtracks. Nested .* on megabyte logs is a denial of service against your own tab. Write linear patterns, compile once, and keep logcat dumps off pastebins. Prototype in the regex tester and apply in the logcat filter.
ECMA-262 versus PCRE
Lookbehind exists in modern JS; possessive quantifiers do not. Do not paste Perl into a JS tester and assume identical results. The regex tester is the JS engine in your browser. That is a feature: you see what production Node or Chromium will do, not what regex101’s PCRE flavor does.
ReDoS on log lines
Log filters should start with tags and levels, not .*message.*. Prefer indexOf for fixed needles. Android threadtime is a good anchored pattern — the worked example is in filter Android logcat in the browser.
const THREADTIME =
/^(\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}\.\d+)\s+(\d+)\s+(\d+)\s+([VDIWEFS])\s+([^:]+):\s?(.*)$/;Flags and lastIndex bugs
Global regexes are stateful. Reusing one /g across lines without lastIndex = 0 skips matches. sticky /y is useful for tokenizers, not whole-line filters. Find and replace has the same footgun if it uses /g on a user pattern.
Workers for 40 MB dumps
Split in a worker, post back counts or matching slices. The UI thread stays for scrolling. Same pattern as chunked hashing. Blob.slice can feed the worker without copying the whole string if you read as text in windows — watch UTF-8 character boundaries.
Secrets in logs
Authorization headers, deep links, SQL. Filter first. If you render matches as HTML, escape them (DOM XSS). Do not innerHTML a log line that contains <script>. Log file analyzer is the mixed-format sibling of logcat.
Tools
Regex tester, logcat filter, log file analyzer, find and replace.
Worked example: filtering a 20 MB logcat without ReDoS
Drop the dump on logcat filter. Do not start with a pattern like (.*ERROR.*){2,} — that is a backtracking bomb. Start with a literal tag (OkHttp) or a level letter. Confirm the threadtime regex matches a sample line in the regex tester with a 100 KB paste, then scale up. If the tab hitchs, you still have a nested quantifier.
Export matching lines as a Blob download. That export is another local file: you can hash it before attaching to a ticket so you know the slice did not change. Do not paste the full dump into a gist. Tokens in query strings should be treated as in the URL encoding article — they are credentials in a log costume.
If you highlight matches in the DOM, use text nodes or escaped HTML. A log line containing <iframe src=javascript:...> is a DOM XSS demo waiting to happen. The filter is a parser, not a renderer. Keep those jobs separate and you get both performance and safety.
Compare this to uploading the dump to a log SaaS: you would get histograms, and they would get every secret in the file. For crash clustering that may be a deliberate vendor choice with a DPA. For “why is this ANR happening on my phone,” local filter plus a 50-line excerpt is enough. The Network-tab audit should show no POST of the 20 MB while you filter.
Frequently asked questions
What is catastrophic backtracking?
When a pattern can match a string many ways, the engine explores them all. (a+)+b on a long string of a’s is the textbook case. Test on the regex tester with a large input before you ship.
Should I compile regex per line?
No. Construct once. Watch lastIndex on /g and /y; reset or you skip lines. The logcat filter should reuse one threadtime pattern.
Is filtering logs in a SaaS analyzer safer?
Usually not. Dumps contain tokens. Filter locally first (logcat in the browser), then share a slice.