Improve performance of SyntaxHighlightFilter

By using Rouge::Lexer.find instead of find_fancy() and memoizing the HTML formatter we can speed up the highlighting process by between 1.7 and 1.8 times (at least when measured using synthetic benchmarks). To measure this I used the following benchmark: require 'benchmark/ips' input = '' Dir['./app/controllers/**/*.rb'].each do |controller| input << <<-EOF <pre><code class="ruby">#{File.read(controller).strip}</code></pre> EOF end document = Nokogiri::HTML.fragment(input) filter = Banzai::Filter::SyntaxHighlightFilter.new(document) puts "Input size: #{(input.bytesize.to_f / 1024).round(2)} KB" Benchmark.ips do |bench| bench.report 'call' do filter.call end end This benchmark produces 250 KB of input. Before these changes the timing output would be as follows: Calculating ------------------------------------- call 1.000 i/100ms ------------------------------------------------- call 22.439 (±35.7%) i/s - 93.000 After these changes the output instead is as follows: Calculating ------------------------------------- call 1.000 i/100ms ------------------------------------------------- call 41.283 (±38.8%) i/s - 148.000 Note that due to the fairly high standard deviation and this being a synthetic benchmark it's entirely possible the real-world improvements are smaller.

Improve performance of SyntaxHighlightFilter
038d6feb · Yorick Peterse · 73772eca · 038d6feb · 038d6feb
Unverified Commit 038d6feb authored Aug 03, 2016 by Yorick Peterse
Show whitespace changes
Inline Side-by-side

Showing with 18 additions and 7 deletions

CHANGELOG CHANGELOG +1 -0

syntax_highlight_filter.rb lib/banzai/filter/syntax_highlight_filter.rb +17 -7

No files found.
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -15,6 +15,7 @@ v 8.11.0 (unreleased)
  - Limit git rev-list output count to one in forced push check
  - Clean up unused routes (Josef Strzibny)
  - Add green outline to New Branch button. !5447 (winniehell)
+  - Improve performance of syntax highlighting Markdown code blocks
  - Update to gitlab_git 10.4.1 and take advantage of preserved Ref objects
  - Retrieve rendered HTML from cache in one request
  - Fix renaming repository when name contains invalid chararacters under project settings

--- a/lib/banzai/filter/syntax_highlight_filter.rb
+++ b/lib/banzai/filter/syntax_highlight_filter.rb
@@ -18,14 +18,11 @@ module Banzai
      def highlight_node(node)
        language = node.attr('class')
        code = node.text
        css_classes = "code highlight"
+        lexer = lexer_for(language)
-        lexer = Rouge::Lexer.find_fancy(language) || Rouge::Lexers::PlainText
-        formatter = Rouge::Formatters::HTML.new
        begin
-          code = formatter.format(lexer.lex(code))
+          code = format(lex(lexer, code))
          css_classes << " js-syntax-highlight #{lexer.tag}"
        rescue
@@ -41,14 +38,27 @@ module Banzai
      private
+      # Separate method so it can be instrumented.
+      def lex(lexer, code)
+        lexer.lex(code)
+      end
+      def format(tokens)
+        rouge_formatter.format(tokens)
+      end
+      def lexer_for(language)
+        (Rouge::Lexer.find(language) || Rouge::Lexers::PlainText).new
+      end
      def replace_parent_pre_element(node, highlighted)
        # Replace the parent `pre` element with the entire highlighted block
        node.parent.replace(highlighted)
      end
      # Override Rouge::Plugins::Redcarpet#rouge_formatter
-      def rouge_formatter(lexer)
+      def rouge_formatter(lexer = nil)
-        Rouge::Formatters::HTML.new
+        @rouge_formatter ||= Rouge::Formatters::HTML.new
      end
    end
  end