BigW Consortium Gitlab

Commit 5830d80b by Yorick Peterse

Pre-calculate Emoji digests

By pre-calculating the digests we can manually construct the emoji URLs, removing the need for using Rails' asset URL helpers. The reason we don't want to use these helpers for Emojis is two-fold: 1. Rails' image_url() method is slow, really slow. For one it _might_ have to calculate digests but it also performs a lot of other intensive operations (judging by the source code and based on measuring timings). 2. We have a lot of Emoji which coupled with the above can result in it taking minutes to load Emoji autocomplete data. Using this pre-calculation setup generating the digests takes around 7 seconds (including the time it takes to start Rails/Rake), and only around 600 milliseconds to load _all_ the autocomplete data of a project (measured locally). This commit _does_ change the Emoji URLs from absolute to relative URLs as these are much easier to generate. To update the Emoji data simply run: rake gemojione:digests Then commit any changes. Fixes gitlab-org/gitlab-ce#14009
parent 8718acdf
...@@ -138,7 +138,7 @@ class ProjectsController < Projects::ApplicationController ...@@ -138,7 +138,7 @@ class ProjectsController < Projects::ApplicationController
participants = ::Projects::ParticipantsService.new(@project, current_user).execute(note_type, note_id) participants = ::Projects::ParticipantsService.new(@project, current_user).execute(note_type, note_id)
@suggestions = { @suggestions = {
emojis: autocomplete_emojis, emojis: AwardEmoji.urls,
issues: autocomplete.issues, issues: autocomplete.issues,
mergerequests: autocomplete.merge_requests, mergerequests: autocomplete.merge_requests,
members: participants members: participants
...@@ -235,17 +235,6 @@ class ProjectsController < Projects::ApplicationController ...@@ -235,17 +235,6 @@ class ProjectsController < Projects::ApplicationController
) )
end end
def autocomplete_emojis
Rails.cache.fetch("autocomplete-emoji-#{Gemojione::VERSION}") do
Emoji.emojis.map do |name, emoji|
{
name: name,
path: view_context.image_url("#{emoji["unicode"]}.png")
}
end
end
end
def repo_exists? def repo_exists?
project.repository_exists? && !project.empty_repo? project.repository_exists? && !project.empty_repo?
end end
......
This source diff could not be displayed because it is too large. You can view the blob instead.
...@@ -48,4 +48,23 @@ class AwardEmoji ...@@ -48,4 +48,23 @@ class AwardEmoji
JSON.parse(File.read(json_path)) JSON.parse(File.read(json_path))
end end
end end
# Returns an Array of Emoji names and their asset URLs.
def self.urls
@urls ||= begin
path = File.join(Rails.root, 'fixtures', 'emojis', 'digests.json')
prefix = Gitlab::Application.config.assets.prefix
digest = Gitlab::Application.config.assets.digest
JSON.parse(File.read(path)).map do |hash|
if digest
fname = "#{hash['unicode']}-#{hash['digest']}"
else
fname = hash['unicode']
end
{ name: hash['name'], path: "#{prefix}/#{fname}.png" }
end
end
end
end end
# This task will generate a standard and Retina sprite of all of the current
# Gemojione Emojis, with the accompanying SCSS map.
#
# It will not appear in `rake -T` output, and the dependent gems are not
# included in the Gemfile by default, because this task will only be needed
# occasionally, such as when new Emojis are added to Gemojione.
begin
require 'sprite_factory'
require 'rmagick'
rescue LoadError
# noop
end
namespace :gemojione do namespace :gemojione do
desc 'Generates Emoji SHA256 digests'
task digests: :environment do
require 'digest/sha2'
require 'json'
dir = Gemojione.index.images_path
digests = AwardEmoji.emojis.map do |name, emoji_hash|
fpath = File.join(dir, "#{emoji_hash['unicode']}.png")
digest = Digest::SHA256.file(fpath).hexdigest
{ name: name, unicode: emoji_hash['unicode'], digest: digest }
end
out = File.join(Rails.root, 'fixtures', 'emojis', 'digests.json')
File.open(out, 'w') do |handle|
handle.write(JSON.pretty_generate(digests))
end
end
# This task will generate a standard and Retina sprite of all of the current
# Gemojione Emojis, with the accompanying SCSS map.
#
# It will not appear in `rake -T` output, and the dependent gems are not
# included in the Gemfile by default, because this task will only be needed
# occasionally, such as when new Emojis are added to Gemojione.
task sprite: :environment do task sprite: :environment do
begin
require 'sprite_factory'
require 'rmagick'
rescue LoadError
# noop
end
check_requirements! check_requirements!
SIZE = 20 SIZE = 20
......
require 'spec_helper'
describe AwardEmoji do
describe '.urls' do
subject { AwardEmoji.urls }
it { is_expected.to be_an_instance_of(Array) }
it { is_expected.to_not be_empty }
context 'every Hash in the Array' do
it 'has the correct keys and values' do
subject.each do |hash|
expect(hash[:name]).to be_an_instance_of(String)
expect(hash[:path]).to be_an_instance_of(String)
end
end
end
end
end
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment