Author Topic: WhatsApp emoji  (Read 32064 times)

0 Members and 1 Guest are viewing this topic.

Offline ThugTopic starter

  • Full Member
  • ***
  • Posts: 237
  • Country: 00
  • Karma: 9
  • Version Info
Re: WhatsApp smiles (need help)
« Reply #30 on: 08 10 2014, 20:49:17 »
alitor,
I believe he meant that now smileys are listed vertically (in columns), but it'd be more natural if they were located horizontally (in rows).
Exactly! Couldn't find the right words ;D

Robyer, "IEview style window" is not the best solution because:
1. If you turn it on, smileys with big resolution will look square in the smiley dialog window (screen 1).
2. You can fix square smileys by increasing SelectionSize value in msl, but that way the window will look huge, and the vertical WindowSize value will decrease to 4 or less smileys despite the value you set (screen 2).
3. And yeah, as watcher said, I don't like the grid.

I just want a simple compact window, like on the last screen, but with smileys sorted horizontally.
« Last Edit: 08 10 2014, 21:26:21 by Thug »
Emojipacks (smileypacks):
WhatsApp, Facebook, Hangouts
 

Offline Robyer

  • Hero Member
  • *****
  • Posts: 1082
  • Country: cz
  • Karma: 60
    • Robyer.cz website
  • Version Info
Re: WhatsApp smiles (need help)
« Reply #31 on: 08 10 2014, 22:01:48 »
Well, it's pretty easy to change order in code, but I'm not sure it's the good solution. It feels like it has own reason to have it sorted this way. Maybe many people are used to it already. But for me it feels weird too, this vertical sorting. So I'm not sure what to do about that...
I was developing mainly Facebook, Omegle, Steam, Dummy and MobileState plugins. Now I'm retired. Goodbye, everyone. ~ You can still find me on Facebook.
 

Offline riki

  • Full Member
  • ***
  • Posts: 101
  • Karma: 0
Re: WhatsApp smiles (need help)
« Reply #32 on: 16 11 2014, 20:00:16 »
sorting apart (no big issue for me personally), smileys work sweetly. THANKS :)
 

Offline Robyer

  • Hero Member
  • *****
  • Posts: 1082
  • Country: cz
  • Karma: 60
    • Robyer.cz website
  • Version Info
Re: WhatsApp smiles (need help)
« Reply #33 on: 16 11 2014, 20:33:50 »
Btw about that sorting someone might add a request to our http://miranda-ng.uservoice.com ;)
I was developing mainly Facebook, Omegle, Steam, Dummy and MobileState plugins. Now I'm retired. Goodbye, everyone. ~ You can still find me on Facebook.
 

Offline watcher

  • Miranda NG support
  • Hero Member
  • *****
  • Posts: 1906
  • Country: il
  • Karma: 157
  • Version Info
Re: WhatsApp smiles (need help)
« Reply #34 on: 13 05 2015, 05:57:26 »
Better be late than sorry - Aliror's smiley pack uploaded to server - http://miranda-ng.org/distr/addons/Smileys/whatsapp.zip. If someone could add it to wiki - it would be great too.
Пожалуйста, внимательно прочтите правила постинга перед тем, как открыть новую тему.
Please read forum rules.
 

Offline ThugTopic starter

  • Full Member
  • ***
  • Posts: 237
  • Country: 00
  • Karma: 9
  • Version Info
Re: WhatsApp smiles (need help)
« Reply #35 on: 13 05 2015, 08:28:32 »
Better be late than sorry - Aliror's smiley pack uploaded to server
Actually original Alitor's pack was incomplete and unsorted. Many smileys lack alternative codes which leads to miranda not displaying smileys received from some official clients (from Nokia OS for sure). I recommend using my updated version (it is continuation of his work, I have credited it in msl).
« Last Edit: 14 05 2015, 12:26:59 by Thug »
Emojipacks (smileypacks):
WhatsApp, Facebook, Hangouts
 

Offline dyrkin

  • Newbie
  • *
  • Posts: 9
  • Karma: 0
Re: WhatsApp smiles (need help)
« Reply #36 on: 13 07 2015, 18:39:58 »
Let me pick it up once again by the following reasons
1. Official Whatsapp pack is not full and has a lot of emoji's missed
2. VK replacement seems good, but sends Unicode escape symbols which are not supported by default even by Whatsapp app (at least in iOS version). On top of that some emojis are still missed
Second point may be resolved by removing &#...; data from msl file, but seems like workaround.

After some surfing found out, that all the work is already done and the only problem is how to reuse it for Miranda NG.
Exhaustive information is available at http://unicode.org/emoji/
Moreover full emoji list with the images for the most protocols is available at http://unicode.org/emoji/charts/full-emoji-list.html

Having the data I see the possible next steps are
1. Enhance SmileyAdd plugins to support data like 'data:image/png;base64,...' as a 'resourcefile' tag. It will allow to use html page mentioned above to easily prepare msl file with desired smiles set
2. Prepare some kind of tool to parse the page and save all the base64 encoded images as pngs and compose msl file automatically.

Basically both tasks are almost the same, it is required to support base64 encoding, but having #1 implemented looks more convenient as a long term solution.

Could somebody criticize text above?
 

Offline Robyer

  • Hero Member
  • *****
  • Posts: 1082
  • Country: cz
  • Karma: 60
    • Robyer.cz website
  • Version Info
Re: WhatsApp smiles (need help)
« Reply #37 on: 14 07 2015, 08:59:17 »
I think we don't need to create base64 support to SmileyAdd, we can use some script/program to convert them to png files, see e.g. http://stackoverflow.com/questions/18108183/online-tool-to-decode-base64-string-to-png
I was developing mainly Facebook, Omegle, Steam, Dummy and MobileState plugins. Now I'm retired. Goodbye, everyone. ~ You can still find me on Facebook.
 

Offline dyrkin

  • Newbie
  • *
  • Posts: 9
  • Karma: 0
Re: WhatsApp smiles (need help)
« Reply #38 on: 14 07 2015, 17:29:41 »
Regarding the program, all the PNGs are already retrieved with one page java code program. Remaining problem is to create msl files.
That is already done also, however needs some additional research. Third argument in "Smiley =" string must be corrected. Probably unicode escape code may be used to compose it.
BTW, code is the following
Code: [Select]
import org.apache.commons.codec.binary.Base64;
import org.apache.commons.io.FileUtils;
import org.jsoup.nodes.Document;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Element;

import java.io.*;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.text.DateFormat;
import java.text.SimpleDateFormat;
import java.util.Date;
import java.util.HashMap;
import java.util.Map;

/**
 * Created by bykov on 13.07.2015.
 */
public class EmojiProcessor {

    private static final Map<String, HashMap <String, String>> protocols = new HashMap<>();
    static {
        protocols.put("BW", new HashMap<String, String>());
        protocols.get("BW").put("name", "BW");
        protocols.get("BW").put("index", "3");
        protocols.put("Apple", new HashMap<String, String>());
        protocols.get("Apple").put("name", "Apple");
        protocols.get("Apple").put("index", "4");
        protocols.put("Twit", new HashMap<String, String>());
        protocols.get("Twit").put("name", "Twit");
        protocols.get("Twit").put("index", "6");
    }

    public static void main(String[] args) throws java.io.IOException{
        //Uncomment to use online version
        //Document doc = Jsoup.connect("http://unicode.org/emoji/charts/full-emoji-list.html").get();
        File input = new File("data/Full Emoji Data.html");
        Document doc = Jsoup.parse(input, "UTF-8");
        String rownumber;
        String character;

        // Clean files and prepare msl files
        for (HashMap protocol: protocols.values()) {
            FileUtils.cleanDirectory(new File("data/" + protocol.get("name").toString() + "/"));
            Files.deleteIfExists(Paths.get("data/" + protocol.get("name").toString() + ".msl"));
            prepareMSLFiles(protocol.get("name").toString());
        }

        // Parse html file extract images and compile msl file
        for (Element table: doc.select("table")) {
            for (Element row : table.select("tr")) {
                // Ignore headers
                if ("Count".equals(row.child(0).text()))
                    continue;
                else {
                    rownumber = row.child(0).text();
                    character = row.child(2).text();
                }
                // For each table record save image and add record to msl file
                for (HashMap protocol: protocols.values()) {
                    if (!row.child((Integer.parseInt(protocol.get("index").toString()))).children().isEmpty()) {
                        savePicture(protocol.get("name").toString(), getImageData(row, Integer.parseInt(protocol.get("index").toString())), rownumber);
                        addMSLRecord(protocol.get("name").toString(),getPNGName(rownumber),character);
                    }
                }
            }
        }
    }

    // Decodes Base64 data tag and save it as png file
    private static void savePicture(String protocol, String img, String name) throws java.io.IOException{
        byte[] data = Base64.decodeBase64(img);
        String filename = "data/" + protocol + "/" + getPNGName(name);
        OutputStream stream = new FileOutputStream(filename);
        stream.write(data);
        stream.close();
    }

    // Remove 'data:image/png;base64,' from the beginning
    private static String getImageData(Element row, int index) {
        return row.child(index).child(0).attr("src").substring(22);
    }

    // Left pad 5 leading zeros to PNG filename
    private static String getPNGName(String name) {
        return String.format("%1$" + 5 + "s", name).replace(' ', '0') + ".png";
    }

    // Add record to msl file for specified picture file and character
    private static void addMSLRecord(String protocol, String pngname, String character) throws java.io.IOException{
        String filename = "data/" + protocol + ".msl";
        OutputStreamWriter stream = new OutputStreamWriter(new FileOutputStream(filename, true), "UTF-8");
        BufferedWriter out = new BufferedWriter(stream);
        out.write("Smiley = \"" + protocol + "\\" + pngname + "\",0,\"" + character + "\"");
        out.newLine();
        out.close();
    }

    // Create msl file and add headers
    private static void prepareMSLFiles(String protocol) throws java.io.IOException{
        DateFormat dateFormat = new SimpleDateFormat("yyyy/MM/dd");
        Date date = new Date();

        String filename = "data/" + protocol + ".msl";
        OutputStreamWriter stream = new OutputStreamWriter(new FileOutputStream(filename, true), "UTF-8");
        BufferedWriter out = new BufferedWriter(stream);
        out.write("Name = \"" + protocol + " Emoji set\""); out.newLine();
        out.write("Author = \"Pavel Bykov\""); out.newLine();
        out.write("Date = \"" + dateFormat.format(date) + "\""); out.newLine();
        out.write("Version = \"1.0\""); out.newLine();
        out.write("SelectionSize = 24, 24"); out.newLine();
        out.write("WindowSize = 10, 8"); out.newLine();
        out.newLine();
        out.close();
    }
}

Current results are also attached for Twitter protocol, they are the smallest )
 

Offline dyrkin

  • Newbie
  • *
  • Posts: 9
  • Karma: 0
Re: WhatsApp smiles (need help)
« Reply #39 on: 14 07 2015, 20:52:51 »
Here is the final version which generates proper msl files. UTF-16 BOM format must have been used.
Code: [Select]
import org.apache.commons.codec.binary.Base64;
import org.apache.commons.io.FileUtils;
import org.apache.commons.lang3.StringEscapeUtils;
import org.jsoup.nodes.Document;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Element;

import java.io.*;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.text.DateFormat;
import java.text.SimpleDateFormat;
import java.util.Date;
import java.util.HashMap;
import java.util.Map;

/**
 * Created by bykov on 13.07.2015.
 */
public class EmojiProcessor {

    private static final Map<String, HashMap <String, String>> protocols = new HashMap<>();
    static {
        protocols.put("BW", new HashMap<String, String>());
        protocols.get("BW").put("name", "BW");
        protocols.get("BW").put("index", "3");
        protocols.put("Apple", new HashMap<String, String>());
        protocols.get("Apple").put("name", "Apple");
        protocols.get("Apple").put("index", "4");
        protocols.put("Twit", new HashMap<String, String>());
        protocols.get("Twit").put("name", "Twit");
        protocols.get("Twit").put("index", "6");
    }

    public static void main(String[] args) throws java.io.IOException{
        //Uncomment to use online version
        //Document doc = Jsoup.connect("http://unicode.org/emoji/charts/full-emoji-list.html").get();
        File input = new File("data/Full Emoji Data.html");
        Document doc = Jsoup.parse(input, "UTF-8");
        String rownumber;
        String character;

        // Clean files and prepare msl files
        for (HashMap protocol: protocols.values()) {
            FileUtils.cleanDirectory(new File("data/" + protocol.get("name").toString() + "/"));
            //Files.deleteIfExists(Paths.get("data/" + protocol.get("name").toString() + ".msl"));
            prepareMSLFiles(protocol.get("name").toString());
        }

        // Parse html file extract images and compile msl file
        for (Element table: doc.select("table")) {
            for (Element row : table.select("tr")) {
                // Ignore headers
                if ("Count".equals(row.child(0).text()))
                    continue;
                else {
                    rownumber = row.child(0).text();
                    character = row.child(1).child(0).attr("name");
                }
                // For each table record save image and add record to msl file
                for (HashMap protocol: protocols.values()) {
                    if (!row.child((Integer.parseInt(protocol.get("index").toString()))).children().isEmpty()) {
                        savePicture(protocol.get("name").toString(), getImageData(row, Integer.parseInt(protocol.get("index").toString())), rownumber);
                        addMSLRecord(protocol.get("name").toString(),getPNGName(rownumber),character);
                    }
                }
            }
        }
    }

    // Decodes Base64 data tag and save it as png file
    private static void savePicture(String protocol, String img, String name) throws java.io.IOException{
        byte[] data = Base64.decodeBase64(img);
        String filename = "data/" + protocol + "/" + getPNGName(name);
        OutputStream stream = new FileOutputStream(filename);
        stream.write(data);
        stream.close();
    }

    // Remove 'data:image/png;base64,' from the beginning
    private static String getImageData(Element row, int index) {
        return row.child(index).child(0).attr("src").substring(22);
    }

    // Left pad 5 leading zeros to PNG filename
    private static String getPNGName(String name) {
        return String.format("%1$" + 5 + "s", name).replace(' ', '0') + ".png";
    }

    // Add record to msl file for specified picture file and character
    private static void addMSLRecord(String protocol, String pngname, String character) throws java.io.IOException{
        String filename = "data/" + protocol + ".msl";
        Writer out = new OutputStreamWriter(new FileOutputStream(filename, true), "UTF-16LE");
        out.append("Smiley = \"" + protocol + "\\" + pngname + "\",0,\"" + returnUnicodeChars(character) + "\"").append("\r\n");
        out.close();
    }

    // Create msl file and add headers
    private static void prepareMSLFiles(String protocol) throws java.io.IOException{
        DateFormat dateFormat = new SimpleDateFormat("yyyy/MM/dd");
        Date date = new Date();

        String filename = "data/" + protocol + ".msl";
        OutputStreamWriter stream = new OutputStreamWriter(new FileOutputStream(filename), "UTF-16LE");
        BufferedWriter out = new BufferedWriter(stream);
        // BOM
        out.write("\ufeff");
        // Header
        out.write("Name = \"" + protocol + " Emoji set\""); out.newLine();
        out.write("Author = \"Pavel Bykov\""); out.newLine();
        out.write("Date = \"" + dateFormat.format(date) + "\""); out.newLine();
        out.write("Version = \"1.0\""); out.newLine();
        out.write("SelectionSize = 24, 24"); out.newLine();
        out.write("WindowSize = 10, 8"); out.newLine();
        out.newLine();
        out.close();
    }

    private static String returnUnicodeChars(String character) {
        String[] chars;
        String returnchar = "";
        chars = character.split("_");
        for (int i=0; i<chars.length; i++) {
            returnchar+=String.valueOf(Character.toChars(Integer.parseInt(chars[i],16)));
        }
        return returnchar;
    }
}

And fully operable Twit archive with PNG and msl files as an example
Full archive with Apple and BW images may be reached at Dropbox https://www.dropbox.com/sh/3xakdhreu66b9yy/AACTIVdwTf_rR50fODah5mBva?dl=0
« Last Edit: 14 07 2015, 21:09:49 by dyrkin »
 

Offline Robyer

  • Hero Member
  • *****
  • Posts: 1082
  • Country: cz
  • Karma: 60
    • Robyer.cz website
  • Version Info
Re: WhatsApp smiles (need help)
« Reply #40 on: 15 07 2015, 19:37:16 »
dyrkin, very nice! What do you think about including the smiley name in the MSL file too? (as image description so it will be showed on mouse hover in smileyadd)
I was developing mainly Facebook, Omegle, Steam, Dummy and MobileState plugins. Now I'm retired. Goodbye, everyone. ~ You can still find me on Facebook.
 

Offline ThugTopic starter

  • Full Member
  • ***
  • Posts: 237
  • Country: 00
  • Karma: 9
  • Version Info
Re: WhatsApp smiles (need help)
« Reply #41 on: 16 07 2015, 06:47:06 »
1. Official Whatsapp pack is not full and has a lot of emoji's missed
The only emoji that are missing (can't be selected in the menu, but incoming/manually entered code transforms into the picture) in the official apps are "Shibuya 109", "!!", and "!?". If you are talking about emoji that are not supported at all by the official apps, there is no reason to include them to the pack, since the purpose of the pack is to provide compability with the official apps. Otherwise it will be not "whatsapp smileypack" but just another custom smileypack.

After some surfing found out, that all the work is already done and the only problem is how to reuse it for Miranda NG.
I can't understand what are you trying to do. All the work is already done and adapted to Miranda.
Emojipacks (smileypacks):
WhatsApp, Facebook, Hangouts
 

Offline dyrkin

  • Newbie
  • *
  • Posts: 9
  • Karma: 0
Re: WhatsApp smiles (need help)
« Reply #42 on: 16 07 2015, 08:44:25 »
including the smiley name in the MSL file too
Reasonable. Will try to extract this also.
The only emoji that are missing
Unfortunately, I'm receiving a lot of smileys from iOS WhatsApp client as squares. Most of them become visible with VK pack, but there is a problem while sending Smileys, since codes like "
 

Offline ThugTopic starter

  • Full Member
  • ***
  • Posts: 237
  • Country: 00
  • Karma: 9
  • Version Info
Re: WhatsApp smiles (need help)
« Reply #43 on: 16 07 2015, 11:59:42 »
Unfortunately, I'm receiving a lot of smileys from iOS WhatsApp client as squares.
What pack do you use? Did you try this one?
« Last Edit: 16 07 2015, 12:03:18 by Thug »
Emojipacks (smileypacks):
WhatsApp, Facebook, Hangouts
 

Offline dyrkin

  • Newbie
  • *
  • Posts: 9
  • Karma: 0
Re: WhatsApp smiles (need help)
« Reply #44 on: 16 07 2015, 12:29:10 »
What pack do you use? Did you try this one?
This one works for most smileys, you are right. There was an error during second part unzipping related to e521.png file, however seems it affects nothing.
Smileys from the wiki plugins page are not working
http://miranda-ng.org/distr/addons/Smileys/whatsapp.zip