favicon_link_tag defaults to /images/favicon.ico, so the location in public was wrong. To avoid 404 errors for user agents blindly requesting /favicon.ico, we add a symlink in public.
../app/assets/images/favicon.ico