As someone who has developed several #ActivityPub software implementations ([Fedify], [Hollo], [BotKit], and [Hackers' Pub]), I believe one of the most frustrating features to implement in the #fediverse is #custom_emoji.
-
@[email protected] You're right, I don't believe there's a specific FEP for custom emojis yet. If one were to be created, accessibility through proper
alttext would definitely need to be a core requirement. The current state where emoji codes like:blob_wave:become the alt text creates significant barriers for screen reader users—especially for non-English speakers, who face an additional language barrier when trying to understand these codes. A proper standard should address how to provide meaningful descriptions that actually convey what the emoji represents in multiple languages.I think we can use
summaryproperty onEmojiobject. It's one of those properties for which the spec allows multi-language mappings:https://www.w3.org/TR/activitystreams-vocabulary/#dfn-summary
-
As someone who has developed several #ActivityPub software implementations (Fedify, Hollo, BotKit, and Hackers' Pub), I believe one of the most frustrating features to implement in the #fediverse is #custom_emoji.
The challenges are numerous:
First, there's no standardization. ActivityPub specifications don't define how custom emoji should work, leading to inconsistent implementations across different servers like Mastodon and Misskey.
Rendering is particularly problematic. Emojis must display properly across different contexts (in text, as reactions, in emoji pickers) while maintaining quality at various sizes. Animated emojis add another layer of complexity.
Perhaps most concerning is the poor #accessibility. Most implementations simply use the emoji code (like
:party_blob:) as thealttext, which provides no meaningful information to screen reader users (in particular, non-English speakers) about what the emoji actually depicts or means.What really dampens my motivation to implement this feature is knowing I'm investing significant effort into something that ultimately creates accessibility barriers. It's disheartening to work hard on a feature that excludes part of the community.
#fedidev
@hongminhee
Fedibird seems to have a feature that lets you see details about emojis, which may also contribute to accessibility.
https://fedibird.com/@noellabo/113876564190790916 -
I think we can use
summaryproperty onEmojiobject. It's one of those properties for which the spec allows multi-language mappings:https://www.w3.org/TR/activitystreams-vocabulary/#dfn-summary
@silverpill @hongminhee @thisismissem
`summary` could represent *some* kind of accessible information, but it's not obvious to me which kind exactly: should it be an accessible name (<https://developer.mozilla.org/en-US/docs/Glossary/Accessible_name>) or an accessible description (<https://developer.mozilla.org/en-US/docs/Glossary/Accessible_description>)? (Or even an HTML `title` attribute?)
(I think it's unfortunate that we need to reinvent an accessibility tree representation, when an AS2 `content` is just an HTML, which has already been equipped with a feature-rich and battle-tested accessibility framework called ARIA, but I digress.)
-
@silverpill @hongminhee @thisismissem
`summary` could represent *some* kind of accessible information, but it's not obvious to me which kind exactly: should it be an accessible name (<https://developer.mozilla.org/en-US/docs/Glossary/Accessible_name>) or an accessible description (<https://developer.mozilla.org/en-US/docs/Glossary/Accessible_description>)? (Or even an HTML `title` attribute?)
(I think it's unfortunate that we need to reinvent an accessibility tree representation, when an AS2 `content` is just an HTML, which has already been equipped with a feature-rich and battle-tested accessibility framework called ARIA, but I digress.)
@tesaguri @silverpill @hongminhee wouldn't it just be the alt text?
-
Requirements in that section are meant for those who want maximum compatibility. For example, JPEGs and images larger than 256 KB are supported by Pleroma and others, but Mastodon doesn't allow them.
Non-square emojis are a bit more complicated. They are not rejected by most servers, but might not be displayed properly by clients.
I just visited your instance and made a screenshot of how emoji is rendered by web UI (attached). The emoji is present, but I can't tell what is depicted there. Some clients enlarge custom emojis on hover, but not Mastodon, so I have to open the image in a new browser tab, or zoom in the page. This is no ideal, hence the recommendation in the FEP.
@silverpill @julian @hongminhee ugh mastodon android squishes this. These are both Mastodon bugs as far as I am concerned. I think as long as you specify what you are making maximum compatibility with that seems OK but long emojos are INCREDIBLY popular on all the misskey and misskey adjacent fediverse areas. So I would argue that usage should be spelled out in the FEP as well.
-
@hongminhee
Fedibird seems to have a feature that lets you see details about emojis, which may also contribute to accessibility.
https://fedibird.com/@noellabo/113876564190790916@hongminhee @silverpill @ozoramore @julian @liaizon @tesaguri
(This is in Japanese. Please use machine translation.)
カスタム絵文字については以前から問題意識があって、Fedibirdの実装を通じて解決策を模索してきました。
FEPを検討しているなら、ぜひ拾い上げてください。ひとまず概要を。
・スクエアではない絵文字の取り扱い
・アクセシビリティと多言語対応
・コピー許可と権利情報
・センシティブの扱い
・画像形式やサイズ、アニメーションの互換性やガイドライン
以下は、カスタム絵文字に付与可能な属性をできるだけ記載したサンプルです。
添付画像は、画像化されたテキストを中心とする絵文字に対し、代替テキストが設定され、ツールチップ表示されているサンプルです。
この絵文字のObject表現（JSON）はこちら。
https://fedibird.com/emojis/prohibits_importing_to_other_servers_ja.json人間向けの公開ページはこちら。
https://fedibird.com/emojis/prohibits_importing_to_other_servers_jaFedibirdのcustom_emojis APIのエンティティ
https://fedibird.com/api/v1/custom_emojis/prohibits_importing_to_other_servers_ja -
@hongminhee @silverpill @ozoramore @julian @liaizon @tesaguri
(This is in Japanese. Please use machine translation.)
カスタム絵文字については以前から問題意識があって、Fedibirdの実装を通じて解決策を模索してきました。
FEPを検討しているなら、ぜひ拾い上げてください。ひとまず概要を。
・スクエアではない絵文字の取り扱い
・アクセシビリティと多言語対応
・コピー許可と権利情報
・センシティブの扱い
・画像形式やサイズ、アニメーションの互換性やガイドライン
以下は、カスタム絵文字に付与可能な属性をできるだけ記載したサンプルです。
添付画像は、画像化されたテキストを中心とする絵文字に対し、代替テキストが設定され、ツールチップ表示されているサンプルです。
この絵文字のObject表現（JSON）はこちら。
https://fedibird.com/emojis/prohibits_importing_to_other_servers_ja.json人間向けの公開ページはこちら。
https://fedibird.com/emojis/prohibits_importing_to_other_servers_jaFedibirdのcustom_emojis APIのエンティティ
https://fedibird.com/api/v1/custom_emojis/prohibits_importing_to_other_servers_ja@hongminhee @silverpill @ozoramore @julian @liaizon @tesaguri When reconsidering the specifications of custom emoji, it would be a good idea to also take into account the information provided by emojis defined in Unicode.
Unicode emojis provide a string that describes what kind of image is depicted, as well as a translation string, so I think it would be better to use them while taking compatibility and consistency into consideration.
-
@hongminhee @silverpill @ozoramore @julian @liaizon @tesaguri When reconsidering the specifications of custom emoji, it would be a good idea to also take into account the information provided by emojis defined in Unicode.
Unicode emojis provide a string that describes what kind of image is depicted, as well as a translation string, so I think it would be better to use them while taking compatibility and consistency into consideration.
@hongminhee @silverpill @ozoramore @julian @liaizon @tesaguri For non-square custom emojis, you should provide width and height in the API response (entity).
This will help client app developers recognize and support non-square emojis.
This is actually very useful, as it improves the user experience by allowing efficient layout before fetching the image.
This may not be necessary for ActivityPub S2S, but it may be useful if you have a case where you can use a dummy image as a placeholder when you can't handle the emoji image.
-
@hongminhee @silverpill @ozoramore @julian @liaizon @tesaguri For non-square custom emojis, you should provide width and height in the API response (entity).
This will help client app developers recognize and support non-square emojis.
This is actually very useful, as it improves the user experience by allowing efficient layout before fetching the image.
This may not be necessary for ActivityPub S2S, but it may be useful if you have a case where you can use a dummy image as a placeholder when you can't handle the emoji image.
@hongminhee @silverpill @ozoramore @julian @liaizon @tesaguri @thisismissem Fedibird currently provides the following information to help users understand the content of custom emojis:
- alternateName (alternate_name)
- ruby
- description
alternateName is not a shortcode that serves as a handle, but an alternative expression or name for the emoji content in text.
ruby is equivalent to the ruby element in HTML. It is also useful as an index when selecting a custom emoji from a picker, especially in CJKV.
-
@hongminhee @silverpill @ozoramore @julian @liaizon @tesaguri @thisismissem Fedibird currently provides the following information to help users understand the content of custom emojis:
- alternateName (alternate_name)
- ruby
- description
alternateName is not a shortcode that serves as a handle, but an alternative expression or name for the emoji content in text.
ruby is equivalent to the ruby element in HTML. It is also useful as an index when selecting a custom emoji from a picker, especially in CJKV.
@hongminhee @silverpill @ozoramore @julian @liaizon @tesaguri @thisismissem description provides detailed information that explains the content of the custom emoji, but it may contain unnecessary content and may not be appropriate for screen readers. alternateName may not provide enough information, but I think it will be sufficient in many cases.
-
@hongminhee @silverpill @ozoramore @julian @liaizon @tesaguri @thisismissem description provides detailed information that explains the content of the custom emoji, but it may contain unnecessary content and may not be appropriate for screen readers. alternateName may not provide enough information, but I think it will be sufficient in many cases.
@hongminhee @silverpill @ozoramore @julian @liaizon @tesaguri @thisismissem The server can register and store information in other languages and provide it based on the user's preferred language.
However, it is difficult to expect all custom emoji providers to fill in this information in multiple languages, so we will have to rely on machine translation to some extent.
-
@hongminhee @silverpill @ozoramore @julian @liaizon @tesaguri @thisismissem The server can register and store information in other languages and provide it based on the user's preferred language.
However, it is difficult to expect all custom emoji providers to fill in this information in multiple languages, so we will have to rely on machine translation to some extent.
@hongminhee @silverpill @ozoramore @julian @liaizon @tesaguri @thisismissem Permission for custom emoji has become a particular issue in recent years.
This calls for custom emoji to be treated as something that the creator retains the rights to and is available under various restrictions, rather than being provided in the public domain where each server can copy and use it.
Traditional custom emoji do not assume conditions of use, so information on these is necessary.
-
@hongminhee @silverpill @ozoramore @julian @liaizon @tesaguri @thisismissem Permission for custom emoji has become a particular issue in recent years.
This calls for custom emoji to be treated as something that the creator retains the rights to and is available under various restrictions, rather than being provided in the public domain where each server can copy and use it.
Traditional custom emoji do not assume conditions of use, so information on these is necessary.
@hongminhee @silverpill @ozoramore @julian @liaizon @tesaguri @thisismissem Fedibird has proposed and implemented several attributes that are considered necessary for conditions of use and licenses. However, when it comes to formulating specifications, the discussion is difficult to reach a consensus, and it is difficult to reach an agreement. Therefore, in addition to these, we have proposed an attribute called copyPermission.
-
@hongminhee @silverpill @ozoramore @julian @liaizon @tesaguri @thisismissem Fedibird has proposed and implemented several attributes that are considered necessary for conditions of use and licenses. However, when it comes to formulating specifications, the discussion is difficult to reach a consensus, and it is difficult to reach an agreement. Therefore, in addition to these, we have proposed an attribute called copyPermission.
@hongminhee @silverpill @ozoramore @julian @liaizon @tesaguri @thisismissem This takes the values allow, deny, and conditional, and provides only information on whether copying is allowed, whether copying is not allowed, and whether copying is allowed only if the conditions of use are confirmed and understood.
We propose that each server implementation should support at least this information as soon as possible, and promptly improve the current situation in which copying is being made against the wishes of custom emoji providers.
-
@hongminhee @silverpill @ozoramore @julian @liaizon @tesaguri @thisismissem This takes the values allow, deny, and conditional, and provides only information on whether copying is allowed, whether copying is not allowed, and whether copying is allowed only if the conditions of use are confirmed and understood.
We propose that each server implementation should support at least this information as soon as possible, and promptly improve the current situation in which copying is being made against the wishes of custom emoji providers.
@hongminhee @silverpill @ozoramore @julian @liaizon @tesaguri @thisismissem We've outlined some of the ideas and implementations we've had in Fedibird so far, as we work to improve support for custom emojis. We hope this will provide a foundation for development and discussion of new features.
-
@hongminhee @silverpill @ozoramore @julian @liaizon @tesaguri @thisismissem We've outlined some of the ideas and implementations we've had in Fedibird so far, as we work to improve support for custom emojis. We hope this will provide a foundation for development and discussion of new features.
@hongminhee @silverpill @ozoramore @julian @liaizon @tesaguri @thisismissem Additional information. Misskey includes free text regarding emoji permission in the object, and information is shared between Misskey users and Fedibird through the federation.
https://misskey.noellabo.jp/emojis/noeskey_misskey_license: {
freeText: ""
}The content is not machine-readable, so administrators must read it and make a judgment.
-
@hongminhee @silverpill @ozoramore @julian @liaizon @tesaguri @thisismissem Additional information. Misskey includes free text regarding emoji permission in the object, and information is shared between Misskey users and Fedibird through the federation.
https://misskey.noellabo.jp/emojis/noeskey_misskey_license: {
freeText: ""
}The content is not machine-readable, so administrators must read it and make a judgment.
@noellabo Very interesting, thank you!
What value goes to
altandtitleattributes when custom emoji is rendered? Is italternateName?@hongminhee @ozoramore @julian @liaizon @tesaguri @thisismissem
-
@noellabo Very interesting, thank you!
What value goes to
altandtitleattributes when custom emoji is rendered? Is italternateName?@hongminhee @ozoramore @julian @liaizon @tesaguri @thisismissem
@silverpill @hongminhee @ozoramore @julian @liaizon @tesaguri @thisismissem When rendering a custom emoji into HTML, it is rendered as <img ... alt=":shortcode:" title="alternateName">. alt is the shortcode, and is the source text. This is so that when a user selects and copies the body of the text, the source text can be obtained from the clipboard.
-
@silverpill @hongminhee @ozoramore @julian @liaizon @tesaguri @thisismissem When rendering a custom emoji into HTML, it is rendered as <img ... alt=":shortcode:" title="alternateName">. alt is the shortcode, and is the source text. This is so that when a user selects and copies the body of the text, the source text can be obtained from the clipboard.
@silverpill @hongminhee @ozoramore @julian @liaizon @tesaguri @thisismissem The same goes for mention anchor tags and URL anchor tags; even if they look like they omit the domain part, or if they clip a long URL, it is important to design them so that the source text can be obtained from the clipboard when copied.
(If you copy the body of mitra, the mention domain will be lost, and the meaning will change.)
-
@silverpill @julian @hongminhee ugh mastodon android squishes this. These are both Mastodon bugs as far as I am concerned. I think as long as you specify what you are making maximum compatibility with that seems OK but long emojos are INCREDIBLY popular on all the misskey and misskey adjacent fediverse areas. So I would argue that usage should be spelled out in the FEP as well.
@liaizon @julian @hongminhee I added a text explaining this additional requirement: https://codeberg.org/silverpill/feps/src/branch/main/9098/fep-9098.md#additional-requirements
Best practices for rendering custom emojis will go into different section (WIP).
