Miranda NG Official Community Forum

Customizing Miranda NG (Addons) => Emoticons (Smileys) => Topic started by: Thug on 03 12 2013, 10:57:06

Title: WhatsApp emojipack (+Apple/Telegram)
Post by: Thug on 03 12 2013, 10:57:06
I have made some research on whatsapp smileys. Since whatsapp uses the emoji, pretty much like Vkontakte does, you can actually take VK smilepack and see most of the incoming smiles. Thats not a solution ofcourse. So, we can easily extract all the smile icons from the official android client (http://www.whatsapp.com/android/current/WhatsApp.apk) - just unpack it with some archiver and you'll find them at res\drawable-hdpi folder. Finding out all the smile codes manually would take forever since there are over 800 smiles, so the main problem is to find the smiles data inside the client. I guess that data can be found in xml files at the same folder with icons, but those xml's are encrypted, I haven't managed to open them. Perhaps someone knows how to open those xmls?

------------------------ Edited ------------------------

Here is the latest version of the emojipack:
Download WhatsApp emojipack 2.2327.4 (https://cloud.mail.ru/public/gVJd/4oBfP9gMh)

Preview

WhatsApp 20x20
(https://i.imgur.com/hGetPfh.jpg)
[close]
WhatsApp 32x32
(https://i.imgur.com/jEOuL8u.jpg)
[close]
WhatsApp 40x40
(https://i.imgur.com/7WDqCUf.jpg)
[close]
WhatsApp 64x64
(https://i.imgur.com/oJSbIFs.jpg)
[close]

Apple 20x20
(https://i.imgur.com/DqsolMM.jpg)
[close]
Apple 32x32
(https://i.imgur.com/W8C8pMD.jpg)
[close]
Apple 40x40
(https://i.imgur.com/6jY99Pn.jpg)
[close]
Apple 64x64
(https://i.imgur.com/N212cz6.jpg)
[close]
[close]
Title: Re:WhatsApp smiles (need help)
Post by: Robyer on 08 12 2013, 21:28:11
I think that smiley code is in filename. It contains hexadecimal number which should be same (or similar) to number representation of emoji character in unicode. Question is in what format are they received/showed in VKontakte chat in Miranda...
Title: Re: WhatsApp smiles (need help)
Post by: watcher on 08 12 2013, 22:29:40
Question is in what format are they received/showed in VKontakte chat in Miranda...


http://miranda-ng.org/distr/addons/Smileys/vk_emoji_local.zip
Title: Re: WhatsApp smiles (need help)
Post by: Robyer on 09 12 2013, 11:07:10

http://miranda-ng.org/distr/addons/Smileys/vk_emoji_local.zip

Oh... so this topic is resolved? :-)

OT: Did someone tried these emojis with Facebook plugin?
Title: Re: WhatsApp smiles (need help)
Post by: Thug on 09 12 2013, 11:10:08
Well, Vkontakte uses 2 (sometimes 3) smiley code formats:
kinda html code: ☝
unicode symboll (emoji): ☝
and simple punctuation marks and characters: :up:

Whatsapp smilies use 1 format for sure - unicode symbols (emoji). You see them when someone sends you a smiley from an official client. But they are not all the same as in VK (50/50 or so).
Title: Re: WhatsApp smiles (need help)
Post by: Thvle on 09 01 2014, 14:58:57
Is possible usage of eomticons in whatsapp plugin?
Title: Re: WhatsApp smiles (need help)
Post by: Thug on 11 01 2014, 00:54:18
It is possible, we just don't have a fully-functional smile pack.
Title: Re: WhatsApp smiles (need help)
Post by: el_even on 06 08 2014, 01:38:50
I hope, that would help (470 WhatsApp smileys, msl file included):
Title: Re: WhatsApp smiles (need help)
Post by: alitor on 14 08 2014, 09:19:07
I hope, that would help (470 WhatsApp smileys, msl file included):
Unfortunately it doesn't work, at least not for me.

Post Merge: 14 08 2014, 18:43:11
Here, today I have compiled this smiley pack. Works quite ok.
One thing though, the order of emoticons is somewhat messy.
And also the pictures themselves are pretty big, so in order to make them look better in the message log one needs to scale them down. Go Options -> Customize -> Smileys. And in "Display" section select "Scale smileys to text height". I've set "Minimum smiley height" to 21.
Title: Re: WhatsApp smiles (need help)
Post by: Thug on 19 08 2014, 09:24:19
Nice work! Did you try some algorithm to convert hexademical picture names to unicode symbols? I have tested your pack and figured out a problem: sending smilies from miranda to of. client works fine, but it is not so fine backwards. You have assigned one code for each smiley, but it looks like whatsapp also uses alternate codes for pretty many of them.

For example:
http://pastebin.com/b4uqx5et

I have assigned both your variant (invisible symbol, you send this) and the one an android official client sends (face). It works fine in both directions. Perhaps it would work fine if you only assign the second one (if all the official clients send only the face), but ideally they should be both listed. And I dunno any easy way to figure out those alternate codes which the official client sends, other then finding out them manually.
Title: Re: WhatsApp smiles (need help)
Post by: alitor on 19 08 2014, 09:41:24
Did you try some algorithm to convert hexademical picture names to unicode symbols?
Well, I've just listed names of the smiley files, pasted them in MS Word and used alt+X on each code to convert them. Unfortunately, I don't have any other info about what combinations are used for each of the smiley in WhatsApp.
Title: Re: WhatsApp smiles (need help)
Post by: Thug on 19 08 2014, 09:46:41
 Actually I have tried to do it manually from scratch, see the progress http://pastebin.com/i2yHnSzh ;D
Title: Re: WhatsApp smiles (need help)
Post by: alitor on 19 08 2014, 09:56:12
Actually I have tried to do it manually from scratch
That's one of the ways, of course, although it's gonna take forever. And I would leave my symbol specifications untouched, adding those other symbols to them rather than replacing them. That would make the pack work both ways with no problem, like you did in the previous post.

Post Merge: 19 08 2014, 10:14:14
I have tested your pack and figured out a problem: sending smilies from miranda to of. client works fine, but it is not so fine backwards.
Just now out of curiosity tried sending a bunch of smileys to Miranda from the official Android app. All of them were displayed correctly.  ???
Title: Re: WhatsApp smiles (need help)
Post by: Thug on 19 08 2014, 13:01:22
Strange. I receive this (I have sent myself all the smileys from the first category).
Title: Re: WhatsApp smiles (need help)
Post by: alitor on 19 08 2014, 13:11:22
Nope, don't have this problem at all.
One thing I forgot to mention when uploading my smiley pack is that when editing the msl file you have to make sure you save it in exactly the same coding system (UTF-16 LE), otherwise it won't work. Maybe this is your case, I don't know. Oh, and I'm using the 64-bit Miranda--not sure if this matters, just in case.
Title: Re: WhatsApp smiles (need help)
Post by: Thug on 19 08 2014, 21:04:27
Oh, and I'm using the 64-bit Miranda--not sure if this matters, just in case.
Perhaps this is the case - I'm using 32 bit version. Didn't check it, but this made me come up with an idea - I have figured out that I only receive smileys with "em_" in their names correctly.

e417.png - undetected
em_1f617.png -ok
em_1f619.png -ok
e105.png - undetected
e409.png - undetected
em_1f61b.png -ok

And so on...
Title: Re: WhatsApp smiles (need help)
Post by: Thug on 22 08 2014, 10:15:35
Weekend, finally :) I have tried x64 version and got the same result. Don't know why the pack works fine for you - it just shouldn't, because the official app sends the alternative codes that are missing in your pack :) Anyway I gonna finish it. I'll try to add the missing codes this weekend.
Title: Re: WhatsApp smiles (need help)
Post by: alitor on 22 08 2014, 10:27:48
Don't know why the pack works fine for you - it just shouldn't
Well, the answer to this question is in the question itself. If everything is the same, you have to look for what is different, and that would be the reason for why it works for me and doesn't work for you.
1) I use the latest official Android app. Did you update yours? Don't you use an iOS app?
2) I run x64 Miranda on Windows 7 x64. Is this the case for you?
3) Did you alter the original pack I've posted? If so, did you save it in the same coding? What program did you use for editing the msl file?
etc
It still works fine for me and my friends using the official android app.
It would be nice if someone else using miranda could also test the pack, in order to get third-party opinion so to say.
Title: Re: WhatsApp smiles (need help)
Post by: Thug on 22 08 2014, 11:54:03
1) Yes, I use the latest official app (on Nokia 305 RM-766)
2) I'm using x32 Miranda on Win 7 x64, but I have tested x64 Miranda too
3) I did't alter your pack

I think the reason is not the receiver (Miranda, OS), but the sender (perhaps the phone model affects how the app works) - for some reason your app sends codes for a group of smileys in softbank format, while as mine sends them all in unified format. I use this table (http://code.iamcal.com/php/emoji/) to find corresponding unified-softbank codes. Anyway both variants are correct and should be present in the pack.
Title: Re: WhatsApp smiles (need help)
Post by: Thug on 24 08 2014, 08:24:27
Have finished it :) See the attachment (just the msl for quick research and the full smilepack). It fully works for me.

P.S. This smiley (http://www.iemoji.com/view/emoji/127/places/shibuya-109-department-store) (e50a.png) bothers me - I haven't found it in my phone client. Not sure if it has an alternative code or not.
Title: Re:WhatsApp smiles (need help)
Post by: mida on 01 09 2014, 08:44:32
Thank you for your work.
but would be easier if example all smilies in the near and not some up, some in the middle and some at the botton of the list
Title: Re: WhatsApp smiles (need help)
Post by: Thug on 03 09 2014, 08:36:22
mida, The smilies are simply sorted by the icon names in alphabetic order :) Only classic yellow round smilies are placed in the beginning. Even so they look quite well sorted. I have thought about sorting them in an order they are presented in the official app , but have abandoned this idea because the SmileyAdd plugin doesn't support division by categories and sorts them in columns rather than lines, which makes it look completely different. Later, perhaps...
Title: Re: WhatsApp smiles (need help)
Post by: mida on 03 09 2014, 18:18:49
for icq smilies i used a white gif on the last line that all real smiles after it on the same position how in original icq7
its not the perfect way but it works and im happy

but here on there many smilies would be nice if one day in future we can use categories example tabs button
Title: Re: WhatsApp smiles (need help)
Post by: Thug on 04 09 2014, 12:32:47
Oh, I see.. I think it would be much easier to slightly change the SmileyAdd plugin to sort smileys in lines, first of all...
Title: Re: WhatsApp smiles (need help)
Post by: Chungalin on 28 09 2014, 12:09:58
I agree that emoji sorting (in lines or columns) should be a plugin option. Or a MSL option.

On the other hand, latest MSL is fine to support both encodings of emoji (Softbank and Unified Unicode), but I think that the first choice should be Unified Unicode. Clients using iOS 5 and later send always Unified Unicode encoding, and I don’t see older clients complain.

References:
http://www.iemoji.com/
http://www.unicode.org/reports/tr51/full-emoji-list.html

According to iemoji, U+E50A is "Shibuya 109 Department Store", and it doesn’t have an Unified Unicode translation. Anyway, looks like it’s no use outside Japan.
Title: Re: WhatsApp smiles (need help)
Post by: Thug on 08 10 2014, 12:41:28
I think that the first choice should be Unified Unicode
I guess we can only know for sure by testing ancient whatsapp clients... And that Shibuya emoji still bothers me :) Will leave it as it is for now, unless someone encounters any problems.

Anyway, I have finally sorted them in an order they are presented in the official apps! :DRINK: Have placed Shibuya 109 icon according to this (http://www.fallensoldiersofwar.com/superdogtrainingtips.com/wp-admin/includes/sentences-using-emoji-icons-5617.png) old iPhone client screenshot cut. "!!" and "!?" emoji, which are also hidden in the official apps, are placed just before red "!" and "?" like in VK smilepack.

I have also created a ticket (http://trac.miranda-ng.org/ticket/815) (in russian, because I was too lazy to write it in english :Р ) about making linear sorting in SmileyAdd plugin.
Title: Re: WhatsApp smiles (need help)
Post by: Robyer on 08 10 2014, 14:58:40
What do you mean by linear sorting?
Title: Re: WhatsApp smiles (need help)
Post by: alitor on 08 10 2014, 17:24:40
What do you mean by linear sorting?
I believe he meant that now smileys are listed vertically (in columns), but it'd be more natural if they were located horizontally (in rows).
Title: Re: WhatsApp smiles (need help)
Post by: Robyer on 08 10 2014, 20:03:33
It can be sorted horizontally - just enable option "IEview style window" in SmileyAdd options ;)
Title: Re: WhatsApp smiles (need help)
Post by: watcher on 08 10 2014, 20:20:59
Robyer,  he doesn't like the looks of it (the grid)
Title: Re: WhatsApp smiles (need help)
Post by: Thug on 08 10 2014, 20:49:17
alitor,
I believe he meant that now smileys are listed vertically (in columns), but it'd be more natural if they were located horizontally (in rows).
Exactly! Couldn't find the right words ;D

Robyer, "IEview style window" is not the best solution because:
1. If you turn it on, smileys with big resolution will look square in the smiley dialog window (screen 1).
2. You can fix square smileys by increasing SelectionSize value in msl, but that way the window will look huge, and the vertical WindowSize value will decrease to 4 or less smileys despite the value you set (screen 2).
3. And yeah, as watcher said, I don't like the grid.

I just want a simple compact window, like on the last screen, but with smileys sorted horizontally.
Title: Re: WhatsApp smiles (need help)
Post by: Robyer on 08 10 2014, 22:01:48
Well, it's pretty easy to change order in code, but I'm not sure it's the good solution. It feels like it has own reason to have it sorted this way. Maybe many people are used to it already. But for me it feels weird too, this vertical sorting. So I'm not sure what to do about that...
Title: Re: WhatsApp smiles (need help)
Post by: riki on 16 11 2014, 20:00:16
sorting apart (no big issue for me personally), smileys work sweetly. THANKS :)
Title: Re: WhatsApp smiles (need help)
Post by: Robyer on 16 11 2014, 20:33:50
Btw about that sorting someone might add a request to our http://miranda-ng.uservoice.com ;)
Title: Re: WhatsApp smiles (need help)
Post by: watcher on 13 05 2015, 05:57:26
Better be late than sorry - Aliror's smiley pack uploaded to server - http://miranda-ng.org/distr/addons/Smileys/whatsapp.zip. If someone could add it to wiki - it would be great too.
Title: Re: WhatsApp smiles (need help)
Post by: Thug on 13 05 2015, 08:28:32
Better be late than sorry - Aliror's smiley pack uploaded to server
Actually original Alitor's pack was incomplete and unsorted. Many smileys lack alternative codes which leads to miranda not displaying smileys received from some official clients (from Nokia OS for sure). I recommend using my updated version (http://forum.miranda-ng.org/index.php?topic=104.msg8449#msg8449) (it is continuation of his work, I have credited it in msl).
Title: Re: WhatsApp smiles (need help)
Post by: dyrkin on 13 07 2015, 18:39:58
Let me pick it up once again by the following reasons
1. Official Whatsapp pack is not full and has a lot of emoji's missed
2. VK replacement seems good, but sends Unicode escape symbols which are not supported by default even by Whatsapp app (at least in iOS version). On top of that some emojis are still missed
Second point may be resolved by removing &#...; data from msl file, but seems like workaround.

After some surfing found out, that all the work is already done and the only problem is how to reuse it for Miranda NG.
Exhaustive information is available at http://unicode.org/emoji/
Moreover full emoji list with the images for the most protocols is available at http://unicode.org/emoji/charts/full-emoji-list.html

Having the data I see the possible next steps are
1. Enhance SmileyAdd plugins to support data like 'data:image/png;base64,...' as a 'resourcefile' tag. It will allow to use html page mentioned above to easily prepare msl file with desired smiles set
2. Prepare some kind of tool to parse the page and save all the base64 encoded images as pngs and compose msl file automatically.

Basically both tasks are almost the same, it is required to support base64 encoding, but having #1 implemented looks more convenient as a long term solution.

Could somebody criticize text above?
Title: Re: WhatsApp smiles (need help)
Post by: Robyer on 14 07 2015, 08:59:17
I think we don't need to create base64 support to SmileyAdd, we can use some script/program to convert them to png files, see e.g. http://stackoverflow.com/questions/18108183/online-tool-to-decode-base64-string-to-png
Title: Re: WhatsApp smiles (need help)
Post by: dyrkin on 14 07 2015, 17:29:41
Regarding the program, all the PNGs are already retrieved with one page java code program. Remaining problem is to create msl files.
That is already done also, however needs some additional research. Third argument in "Smiley =" string must be corrected. Probably unicode escape code may be used to compose it.
BTW, code is the following
Code: [Select]
import org.apache.commons.codec.binary.Base64;
import org.apache.commons.io.FileUtils;
import org.jsoup.nodes.Document;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Element;

import java.io.*;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.text.DateFormat;
import java.text.SimpleDateFormat;
import java.util.Date;
import java.util.HashMap;
import java.util.Map;

/**
 * Created by bykov on 13.07.2015.
 */
public class EmojiProcessor {

    private static final Map<String, HashMap <String, String>> protocols = new HashMap<>();
    static {
        protocols.put("BW", new HashMap<String, String>());
        protocols.get("BW").put("name", "BW");
        protocols.get("BW").put("index", "3");
        protocols.put("Apple", new HashMap<String, String>());
        protocols.get("Apple").put("name", "Apple");
        protocols.get("Apple").put("index", "4");
        protocols.put("Twit", new HashMap<String, String>());
        protocols.get("Twit").put("name", "Twit");
        protocols.get("Twit").put("index", "6");
    }

    public static void main(String[] args) throws java.io.IOException{
        //Uncomment to use online version
        //Document doc = Jsoup.connect("http://unicode.org/emoji/charts/full-emoji-list.html").get();
        File input = new File("data/Full Emoji Data.html");
        Document doc = Jsoup.parse(input, "UTF-8");
        String rownumber;
        String character;

        // Clean files and prepare msl files
        for (HashMap protocol: protocols.values()) {
            FileUtils.cleanDirectory(new File("data/" + protocol.get("name").toString() + "/"));
            Files.deleteIfExists(Paths.get("data/" + protocol.get("name").toString() + ".msl"));
            prepareMSLFiles(protocol.get("name").toString());
        }

        // Parse html file extract images and compile msl file
        for (Element table: doc.select("table")) {
            for (Element row : table.select("tr")) {
                // Ignore headers
                if ("Count".equals(row.child(0).text()))
                    continue;
                else {
                    rownumber = row.child(0).text();
                    character = row.child(2).text();
                }
                // For each table record save image and add record to msl file
                for (HashMap protocol: protocols.values()) {
                    if (!row.child((Integer.parseInt(protocol.get("index").toString()))).children().isEmpty()) {
                        savePicture(protocol.get("name").toString(), getImageData(row, Integer.parseInt(protocol.get("index").toString())), rownumber);
                        addMSLRecord(protocol.get("name").toString(),getPNGName(rownumber),character);
                    }
                }
            }
        }
    }

    // Decodes Base64 data tag and save it as png file
    private static void savePicture(String protocol, String img, String name) throws java.io.IOException{
        byte[] data = Base64.decodeBase64(img);
        String filename = "data/" + protocol + "/" + getPNGName(name);
        OutputStream stream = new FileOutputStream(filename);
        stream.write(data);
        stream.close();
    }

    // Remove 'data:image/png;base64,' from the beginning
    private static String getImageData(Element row, int index) {
        return row.child(index).child(0).attr("src").substring(22);
    }

    // Left pad 5 leading zeros to PNG filename
    private static String getPNGName(String name) {
        return String.format("%1$" + 5 + "s", name).replace(' ', '0') + ".png";
    }

    // Add record to msl file for specified picture file and character
    private static void addMSLRecord(String protocol, String pngname, String character) throws java.io.IOException{
        String filename = "data/" + protocol + ".msl";
        OutputStreamWriter stream = new OutputStreamWriter(new FileOutputStream(filename, true), "UTF-8");
        BufferedWriter out = new BufferedWriter(stream);
        out.write("Smiley = \"" + protocol + "\\" + pngname + "\",0,\"" + character + "\"");
        out.newLine();
        out.close();
    }

    // Create msl file and add headers
    private static void prepareMSLFiles(String protocol) throws java.io.IOException{
        DateFormat dateFormat = new SimpleDateFormat("yyyy/MM/dd");
        Date date = new Date();

        String filename = "data/" + protocol + ".msl";
        OutputStreamWriter stream = new OutputStreamWriter(new FileOutputStream(filename, true), "UTF-8");
        BufferedWriter out = new BufferedWriter(stream);
        out.write("Name = \"" + protocol + " Emoji set\""); out.newLine();
        out.write("Author = \"Pavel Bykov\""); out.newLine();
        out.write("Date = \"" + dateFormat.format(date) + "\""); out.newLine();
        out.write("Version = \"1.0\""); out.newLine();
        out.write("SelectionSize = 24, 24"); out.newLine();
        out.write("WindowSize = 10, 8"); out.newLine();
        out.newLine();
        out.close();
    }
}

Current results are also attached for Twitter protocol, they are the smallest )
Title: Re: WhatsApp smiles (need help)
Post by: dyrkin on 14 07 2015, 20:52:51
Here is the final version which generates proper msl files. UTF-16 BOM format must have been used.
Code: [Select]
import org.apache.commons.codec.binary.Base64;
import org.apache.commons.io.FileUtils;
import org.apache.commons.lang3.StringEscapeUtils;
import org.jsoup.nodes.Document;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Element;

import java.io.*;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.text.DateFormat;
import java.text.SimpleDateFormat;
import java.util.Date;
import java.util.HashMap;
import java.util.Map;

/**
 * Created by bykov on 13.07.2015.
 */
public class EmojiProcessor {

    private static final Map<String, HashMap <String, String>> protocols = new HashMap<>();
    static {
        protocols.put("BW", new HashMap<String, String>());
        protocols.get("BW").put("name", "BW");
        protocols.get("BW").put("index", "3");
        protocols.put("Apple", new HashMap<String, String>());
        protocols.get("Apple").put("name", "Apple");
        protocols.get("Apple").put("index", "4");
        protocols.put("Twit", new HashMap<String, String>());
        protocols.get("Twit").put("name", "Twit");
        protocols.get("Twit").put("index", "6");
    }

    public static void main(String[] args) throws java.io.IOException{
        //Uncomment to use online version
        //Document doc = Jsoup.connect("http://unicode.org/emoji/charts/full-emoji-list.html").get();
        File input = new File("data/Full Emoji Data.html");
        Document doc = Jsoup.parse(input, "UTF-8");
        String rownumber;
        String character;

        // Clean files and prepare msl files
        for (HashMap protocol: protocols.values()) {
            FileUtils.cleanDirectory(new File("data/" + protocol.get("name").toString() + "/"));
            //Files.deleteIfExists(Paths.get("data/" + protocol.get("name").toString() + ".msl"));
            prepareMSLFiles(protocol.get("name").toString());
        }

        // Parse html file extract images and compile msl file
        for (Element table: doc.select("table")) {
            for (Element row : table.select("tr")) {
                // Ignore headers
                if ("Count".equals(row.child(0).text()))
                    continue;
                else {
                    rownumber = row.child(0).text();
                    character = row.child(1).child(0).attr("name");
                }
                // For each table record save image and add record to msl file
                for (HashMap protocol: protocols.values()) {
                    if (!row.child((Integer.parseInt(protocol.get("index").toString()))).children().isEmpty()) {
                        savePicture(protocol.get("name").toString(), getImageData(row, Integer.parseInt(protocol.get("index").toString())), rownumber);
                        addMSLRecord(protocol.get("name").toString(),getPNGName(rownumber),character);
                    }
                }
            }
        }
    }

    // Decodes Base64 data tag and save it as png file
    private static void savePicture(String protocol, String img, String name) throws java.io.IOException{
        byte[] data = Base64.decodeBase64(img);
        String filename = "data/" + protocol + "/" + getPNGName(name);
        OutputStream stream = new FileOutputStream(filename);
        stream.write(data);
        stream.close();
    }

    // Remove 'data:image/png;base64,' from the beginning
    private static String getImageData(Element row, int index) {
        return row.child(index).child(0).attr("src").substring(22);
    }

    // Left pad 5 leading zeros to PNG filename
    private static String getPNGName(String name) {
        return String.format("%1$" + 5 + "s", name).replace(' ', '0') + ".png";
    }

    // Add record to msl file for specified picture file and character
    private static void addMSLRecord(String protocol, String pngname, String character) throws java.io.IOException{
        String filename = "data/" + protocol + ".msl";
        Writer out = new OutputStreamWriter(new FileOutputStream(filename, true), "UTF-16LE");
        out.append("Smiley = \"" + protocol + "\\" + pngname + "\",0,\"" + returnUnicodeChars(character) + "\"").append("\r\n");
        out.close();
    }

    // Create msl file and add headers
    private static void prepareMSLFiles(String protocol) throws java.io.IOException{
        DateFormat dateFormat = new SimpleDateFormat("yyyy/MM/dd");
        Date date = new Date();

        String filename = "data/" + protocol + ".msl";
        OutputStreamWriter stream = new OutputStreamWriter(new FileOutputStream(filename), "UTF-16LE");
        BufferedWriter out = new BufferedWriter(stream);
        // BOM
        out.write("\ufeff");
        // Header
        out.write("Name = \"" + protocol + " Emoji set\""); out.newLine();
        out.write("Author = \"Pavel Bykov\""); out.newLine();
        out.write("Date = \"" + dateFormat.format(date) + "\""); out.newLine();
        out.write("Version = \"1.0\""); out.newLine();
        out.write("SelectionSize = 24, 24"); out.newLine();
        out.write("WindowSize = 10, 8"); out.newLine();
        out.newLine();
        out.close();
    }

    private static String returnUnicodeChars(String character) {
        String[] chars;
        String returnchar = "";
        chars = character.split("_");
        for (int i=0; i<chars.length; i++) {
            returnchar+=String.valueOf(Character.toChars(Integer.parseInt(chars[i],16)));
        }
        return returnchar;
    }
}

And fully operable Twit archive with PNG and msl files as an example
Full archive with Apple and BW images may be reached at Dropbox https://www.dropbox.com/sh/3xakdhreu66b9yy/AACTIVdwTf_rR50fODah5mBva?dl=0
Title: Re: WhatsApp smiles (need help)
Post by: Robyer on 15 07 2015, 19:37:16
dyrkin, very nice! What do you think about including the smiley name in the MSL file too? (as image description so it will be showed on mouse hover in smileyadd)
Title: Re: WhatsApp smiles (need help)
Post by: Thug on 16 07 2015, 06:47:06
1. Official Whatsapp pack is not full and has a lot of emoji's missed
The only emoji that are missing (can't be selected in the menu, but incoming/manually entered code transforms into the picture) in the official apps are "Shibuya 109", "!!", and "!?". If you are talking about emoji that are not supported at all by the official apps, there is no reason to include them to the pack, since the purpose of the pack is to provide compability with the official apps. Otherwise it will be not "whatsapp smileypack" but just another custom smileypack.

After some surfing found out, that all the work is already done and the only problem is how to reuse it for Miranda NG.
I can't understand what are you trying to do. All the work is already done and adapted to Miranda.
Title: Re: WhatsApp smiles (need help)
Post by: dyrkin on 16 07 2015, 08:44:25
including the smiley name in the MSL file too
Reasonable. Will try to extract this also.
The only emoji that are missing
Unfortunately, I'm receiving a lot of smileys from iOS WhatsApp client as squares. Most of them become visible with VK pack, but there is a problem while sending Smileys, since codes like "
Title: Re: WhatsApp smiles (need help)
Post by: Thug on 16 07 2015, 11:59:42
Unfortunately, I'm receiving a lot of smileys from iOS WhatsApp client as squares.
What pack do you use? Did you try this one (http://forum.miranda-ng.org/index.php?topic=104.msg8449#msg8449)?
Title: Re: WhatsApp smiles (need help)
Post by: dyrkin on 16 07 2015, 12:29:10
What pack do you use? Did you try this one (http://forum.miranda-ng.org/index.php?topic=104.msg8449#msg8449)?
This one works for most smileys, you are right. There was an error during second part unzipping related to e521.png file, however seems it affects nothing.
Smileys from the wiki plugins page are not working
http://miranda-ng.org/distr/addons/Smileys/whatsapp.zip
Title: Re: WhatsApp smiles (need help)
Post by: Thug on 16 07 2015, 12:57:28
This one works for most smileys, you are right.
Should work for 100% :) Please report if you find any issues.

There was an error during second part unzipping related to e521.png file, however seems it affects nothing.
This is multi-volume archive (I had to split it due to forum limitations). Such archives should be opened by placing all the parts to the same folder and using only the first part. The error occures while trying to unzip other parts separately, no need to do it...

Smileys from the wiki plugins page are not working
Ofcourse they aren't - I've told about it (http://forum.miranda-ng.org/index.php?topic=104.msg12151#msg12151) just before your fist post in this thread ???

watcher, please, update the smileypack on the server...
Title: Re:WhatsApp smiles (need help)
Post by: dyrkin on 18 07 2015, 13:52:47
BTW, here (https://www.dropbox.com/s/s50e7xhz9ksreyy/Emoji.7z?dl=0) is a full Emoji set grabbed from the official page (http://unicode.org/emoji/charts/full-emoji-list.html). With tooltips :)
Feel free to use, share, or whatever you want..

Traditionally, here is the code used for files generation
Code: [Select]
import org.apache.commons.codec.binary.Base64;
import org.apache.commons.io.FileUtils;
import org.jsoup.nodes.Document;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Element;

import java.io.*;
import java.text.DateFormat;
import java.text.SimpleDateFormat;
import java.util.Date;
import java.util.HashMap;
import java.util.Map;

/**
 * Created by bykov on 13.07.2015.
 */
public class EmojiProcessor {

    private static final String outDir = "data/out/";

    private static final Map<String, HashMap <String, String>> protocols = new HashMap<>();
    static {
        protocols.put("BW", new HashMap<String, String>());
        protocols.get("BW").put("name", "BW");
        protocols.get("BW").put("index", "3");
        protocols.put("Apple", new HashMap<String, String>());
        protocols.get("Apple").put("name", "Apple");
        protocols.get("Apple").put("index", "4");
        protocols.put("Andr", new HashMap<String, String>());
        protocols.get("Andr").put("name", "Andr");
        protocols.get("Andr").put("index", "5");
        protocols.put("Twit", new HashMap<String, String>());
        protocols.get("Twit").put("name", "Twit");
        protocols.get("Twit").put("index", "6");
        protocols.put("Wind", new HashMap<String, String>());
        protocols.get("Wind").put("name", "Wind");
        protocols.get("Wind").put("index", "7");
        protocols.put("GMail", new HashMap<String, String>());
        protocols.get("GMail").put("name", "GMail");
        protocols.get("GMail").put("index", "8");
        protocols.put("DCM", new HashMap<String, String>());
        protocols.get("DCM").put("name", "DCM");
        protocols.get("DCM").put("index", "9");
        protocols.put("KDDI", new HashMap<String, String>());
        protocols.get("KDDI").put("name", "KDDI");
        protocols.get("KDDI").put("index", "10");
        protocols.put("SB", new HashMap<String, String>());
        protocols.get("SB").put("name", "SB");
        protocols.get("SB").put("index", "11");
    }

    public static void main(String[] args) throws java.io.IOException{
        //Uncomment to use online version
        //Document doc = Jsoup.connect("http://unicode.org/emoji/charts/full-emoji-list.html").get();
        File input = new File("data/Full Emoji Data.html");
        Document doc = Jsoup.parse(input, "UTF-8");
        String rownumber;
        String character;
        String tooltip;

        // Clean files and prepare msl files
        for (HashMap protocol: protocols.values()) {
            FileUtils.deleteDirectory(new File(outDir + protocol.get("name").toString() + "/"));
            new File(outDir + protocol.get("name").toString()).mkdir();
            prepareMSLFiles(protocol.get("name").toString());
        }

        // Parse html file extract images and compile msl file
        for (Element table: doc.select("table")) {
            for (Element row : table.select("tr")) {
                // Ignore headers
                if ("Count".equals(row.child(0).text()))
                    continue;
                else {
                    rownumber = row.child(0).text();
                    character = row.child(1).child(0).attr("name");
                    tooltip = row.child(12).text();
                }
                // For each table record save image and add record to msl file
                for (HashMap protocol: protocols.values()) {
                    if (!row.child((Integer.parseInt(protocol.get("index").toString()))).children().isEmpty()) {
                        savePicture(protocol.get("name").toString(), getImageData(row, Integer.parseInt(protocol.get("index").toString())), rownumber);
                        addMSLRecord(protocol.get("name").toString(),getPNGName(rownumber),character, tooltip);
                    }
                }
            }
        }
    }

    // Decode Base64 data tag and save it as png file
    private static void savePicture(String protocol, String img, String name) throws java.io.IOException{
        byte[] data = Base64.decodeBase64(img);
        String filename = outDir + protocol + "/" + getPNGName(name);
        OutputStream stream = new FileOutputStream(filename);
        stream.write(data);
        stream.close();
    }

    // Remove 'data:image/png;base64,' from the beginning
    private static String getImageData(Element row, int index) {
        return row.child(index).child(0).attr("src").substring(22);
    }

    // Left pad 5 leading zeros to PNG filename
    private static String getPNGName(String name) {
        return String.format("%1$" + 5 + "s", name).replace(' ', '0') + ".png";
    }

    // Add record to msl file for specified picture file and character
    private static void addMSLRecord(String protocol, String pngname, String character, String tooltip) throws java.io.IOException{
        String filename = outDir + protocol + ".msl";
        Writer out = new OutputStreamWriter(new FileOutputStream(filename, true), "UTF-16LE");
        out.append("Smiley = \"")
                .append(protocol)
                .append("\\")
                .append(pngname)
                .append("\",0,\"")
                .append(returnUnicodeChars(character))
                .append("\",\"")
                .append(tooltip)
                .append("\"\r\n");
        out.close();
    }

    // Create msl file and add headers
    private static void prepareMSLFiles(String protocol) throws java.io.IOException{
        DateFormat dateFormat = new SimpleDateFormat("yyyy/MM/dd");
        Date date = new Date();

        String filename = outDir + protocol + ".msl";
        OutputStreamWriter stream = new OutputStreamWriter(new FileOutputStream(filename), "UTF-16LE");
        BufferedWriter out = new BufferedWriter(stream);
        // BOM
        out.write("\ufeff");
        // Header
        out.write("Name = \"" + protocol + " Emoji set\""); out.newLine();
        out.write("Author = \"Pavel Bykov\""); out.newLine();
        out.write("Date = \"" + dateFormat.format(date) + "\""); out.newLine();
        out.write("Version = \"1.0\""); out.newLine();
        out.write("SelectionSize = 24, 24"); out.newLine();
        out.write("WindowSize = 10, 8"); out.newLine();
        out.newLine();
        out.close();
    }

    private static String returnUnicodeChars(String character) {
        String[] chars;
        String returnchar = "";
        chars = character.split("_");
        for (int i=0; i<chars.length; i++) {
            returnchar+=String.valueOf(Character.toChars(Integer.parseInt(chars[i],16)));
        }
        return returnchar;
    }
}

Thank you
Title: Re: WhatsApp smiles (need help)
Post by: Thug on 05 10 2015, 15:08:19
Hi. First of all, thanks a lot to MikalaiR for implementing horizontal sorting in SmileyAdd plugin, looks much better now :THUMBS UP:

And now, behold a new version (1.3) of whatsapp smilepack! Here are the changes:
 - multicultural emoji (http://www.whatsappforall.com/whatsapp-for-android-adds-racial-emojis/) are added, including new emoji "Spock" and "Middle Finger". These new emoji are united in 59 six-icon pictures in android client sources, so, due to the lack of plugin functionality, I had to just split them in graphics redactor.
 - updated the flags (http://www.whatsappforall.com/whatsapp-for-android-redesign-and-added-flags/) (new Pakistan and Bangladesh flags were added, the rest were redesigned).

There are 1191 emoji in the pack now, and looks like will be much more in the future. As you can see, it's getting pretty hard to navigate through it. New functions such as tabs and that selection window for multicultural emoji are needed.

Btw, I have noticed that some odd symbol comes from iPhones with a group of emoji. The symbol is "FE0F" in hexademical code. And the emoji are, for example, horoscope signs, a snowflake, and many others. I wonder what is the purpose of it :-\
Title: Re: WhatsApp smiles (need help)
Post by: watcher on 05 10 2015, 17:02:48
Thug,  can't download the pack for updating on server :( "Page not found" error.
Title: Re: WhatsApp smiles (need help)
Post by: Thug on 05 10 2015, 21:22:44
Thug,  can't download the pack for updating on server  "Page not found" error.
Sorry, made an attachment. Thought of using VK as a file hosting and failed ;D Thanks!
Title: Re: WhatsApp smiles (need help)
Post by: watcher on 05 10 2015, 21:32:45
The pack has been updated (http://miranda-ng.org/distr/addons/Smileys/WhatsApp/).
Title: Re: WhatsApp emoji
Post by: Thug on 08 12 2016, 11:59:11
Hi. I had stopped updating this pack almost a year ago - when whatsapp plugin became completely unusable. But I was researching the web version of whatsapp recently and found all the source data requiered to build a new version of the smileypack, so I decided to undertake it again :)

The new version contains all currently supported 1770 emoji (+1 strange icon without a code) extracted from the web version as well as two discontinued emoji (Shibuya (http://emojipedia.org/shibuya/) and Olympic Rings (http://emojipedia.org/olympic-rings/) - they still work in some old clients). Now they are available in two resolutions: 40x40 and 64x64.

I have hidden all the multicultural (different skin color) emoji and the ones located in the last section ("Other") to make the selection menu look nicer and in order to match the official presentation. Feel free to unhide them by removing "*" symbol in "Smiley*" at the beginning of the strings.
* Thanks to Wishmaster for the tip!

The msl is saved in a simple UTF-8 format.
Dunno why did we use UTF-16 LE ???

I could also make an online version (just an .msl with direct links to the icons on whatsapp server), but does anyone really need it? Note that whatsapp developers regulary change everything and the links can turn broken anytime. Anyway, if you are interested, the links look like this:
https://web.whatsapp.com/img/c9c24f56.png

Download WhatsApp emoji pack v. 2.0 (https://cloud.mail.ru/public/M6ci/MqrVroRkz)

Hope it will inspire someone to continue developing whatsapp plugin or at least will come in handy for something :THUMBS UP:
Btw, it is 90% compatible with VK protocol now.
Title: Re: WhatsApp emoji
Post by: Thug on 22 02 2017, 10:54:10
Catching up with WhatsApp implementing iOS 10.2 emojis (http://blog.emojipedia.org/ios-10-2-emoji-changelog/) (added support for Unicode 9 emojis, as well as new professions and a graphical redesign).

Download WhatsApp emoji pack v. 2.1 (https://cloud.mail.ru/public/LCUd/C2hww5upo)
Title: Re: WhatsApp emoji
Post by: dartraiden on 29 11 2017, 12:39:49
WhatsApp plugin is deprecated.
Title: Re: WhatsApp emoji
Post by: Thug on 22 09 2018, 19:02:35
Hi. I know that WhatsApp protocol is not supported right now. But sloppdawop (https://forum.miranda-ng.org/index.php?action=profile;u=1110) asked me to include downsized images in the next release, so I have made the next release :)

Download:
WhatsApp EmojiPack 2.17 (https://cloud.mail.ru/public/9W2f/QYu4j9BUV)

;  Version 2.17 (2018.09.22):
;     - Emojis from Unicode 10.0 and new flags are added.
;     - New WhatsApp emoji designs. The pack now includes both versions of the emoji designs
;       currently supported by WhatsApp: classic Apple designs (a) and WhatsApp custom designs (https://blog.emojipedia.org/whatsapp-releases-its-own-emoji-set/) (w)
;       (some emojis like Texas flag are missing from the Apple emoji set for now).
;     - Added downsized resolutions (32x32 and 20x20).
;     - The emojipack version number is aligned with Android WhatsApp version.

While researching whatsapp resources I have noticed that it stores both Apple and new custom WhatsApp emojis in the web and windows versions. If you use iPhone, the web or windows version will show you apple-style emojis, while using any other phone to connect via PC will trigger WhatsApp custom emoji set. The Apple version is missing some emojis, but iOS 12 already has some of them (like the Texas flag). Anyway, I'm a bit (about a year) late with this version. I guess you will see a new WhatsApp version in some days or weeks that will introduce a huge emoji update.
Btw, you can find an unused image e2364.png (https://web.whatsapp.com/img/88179f049c0d1d26c823d2954ecb54a7_a_e2364-64.png) in the "a" version of the emojipack. If you are curious, it's just a censored version of Taiwanese flag (https://emojipedia.org/flag-for-taiwan/) for Chinese region of iPhone users.

And I still hope that someone will bring back WhatsApp to Miranda NG. There is a bunch of working unofficial clients (https://www.downloadappsmobile.com/) already, but no one for Windows, what is strange.
Title: WhatsApp/Apple/Telegram
Post by: Thug on 01 07 2023, 04:16:41
Давно пора обновить смайлпак :)
Сделал новую версию на основе актуальной веб-версии вацапа, заменил там только иконки для самых новых эмодзи (https://blog.emojipedia.org/ios-16-4-emoji-changelog/) в Apple наборе, потому что в веб-версии там до сих пор используются не эппловские картинки (это косяк веб-версии вацапа, который они должны будут устранить рано или поздно).
Напоминаю, что в веб-версии вацапа присутствуют два набора эмодзи - набор с собственным дизайном для пользователей на андройде, и набор с дизайном от Apple для пользователей на iOS (изначально он использовался во всех версиях вацапа на всех устройствах). Поэтому в этом смайлпаке присутствуют оба набора - выбирайте, какой хотите.

Скачать WhatsApp emojipack 2.2327.4 (https://cloud.mail.ru/public/gVJd/4oBfP9gMh)
Отдельно загружаю версию 16x16 (https://cloud.mail.ru/public/v4vS/7NFUEVszf), кому надо.

Скриншоты окна выбора смайлов

WhatsApp 20x20
(https://i.imgur.com/hGetPfh.jpg)
[close]
WhatsApp 32x32
(https://i.imgur.com/jEOuL8u.jpg)
[close]
WhatsApp 40x40
(https://i.imgur.com/7WDqCUf.jpg)
[close]
WhatsApp 64x64
(https://i.imgur.com/oJSbIFs.jpg)
[close]

Apple 20x20
(https://i.imgur.com/DqsolMM.jpg)
[close]
Apple 32x32
(https://i.imgur.com/W8C8pMD.jpg)
[close]
Apple 40x40
(https://i.imgur.com/6jY99Pn.jpg)
[close]
Apple 64x64
(https://i.imgur.com/N212cz6.jpg)
[close]
[close]

Что касается телеграма - делать для него отдельный смайлпак не вижу смысла, так как там используется дизайн от Apple. Можете смело использовать в телеграме набор Apple из этого пака.
Title: Re: WhatsApp emojipack (+Apple/Telegram)
Post by: dartraiden on 28 07 2023, 19:37:01
Thug, если сделаете скриншот (окна со смайлами, вылезающими при нажатии "Тест" в настройках SmileyAdd), то я добавлю ваши смайлы на сайт.
Title: WhatsApp/Apple/Telegram
Post by: Thug on 29 07 2023, 02:44:02
dartraiden, добавил в предыдущий пост под спойлер  :THUMBS UP: