Miranda NG Official Community Forum
Forum for English speaking Miranda NG users => Bug reports => Topic started by: Smiling Spectre on 06 09 2018, 19:56:52
-
Today at evening Miranda updated, and got Db_autobackups.dll of 11:36 06.09.2018. (Version 0.1.0.1).
Immediately autobackups stopped to work. No more backups, even if pressing "backup now" button. And regardless of that, every minute autobackup window flashes momentarily (Miranda set to autobackup once per day or on exit).
Something definitely broken now.
Update: seems, base got broken during update. I cannot compact or edit it too. *sigh* Old format was much less fragile...
-
if you used 0.95.8, you had already converted your db into new format, so you could have problems with that initial format
you can try to create an empty profile and import the 'broken' one into it, then enable auto backup again
seems, base got broken during update
no, there's no database update during migration from 0.95.8 to .9
Old format was much less fragile...
the problem is that we rolled out 0.95.8 using libmdbx 0.9, which had some problems. now we're using 1.0.2, which is much more solid. so once imported from the old one, your profile will have no problems (hopefully)
-
if you used 0.95.8, you had already converted your db into new format, so you could have problems with that initial format
Well, base updated yesterday without any problems. I worked with Miranda all day. Today evening it updated again, to 0.95.9 build #20730. And something happened, it seems.
you can try to create an empty profile and import the 'broken' one into it, then enable auto backup again
How to create empty profile? I tried to remove old file from Profiles folder. It seems worked, but on import I got something new.
---------------------------
Microsoft Visual C++ Runtime Library
---------------------------
Assertion failed!
Program: <program name unknown>
File: mdbx_cmp_int_a2
Line: 6051
Expression: a->iov_len == b->iov_len
For information on how your program can cause an assertion
failure, see the Visual C++ documentation on asserts
(Press Retry to debug the application - JIT must be enabled)
Funny...
It happens on full import or "all contacts and messages", at stage "History import".
If I import contacts only, it works (without history, of course).
- If I import manually, set everything, except contacts and system history, it imports nothing at all.
- If I import system history only, it imports 14 messages (for 5 years, uh?).
- If I import contacts only, it does nothing (maybe because I imported everything before?)
- If I import contacts AND system history, it tries to import contact again, and even adds 24 more. %)
- And if I set Contacts AND History, it crashes just as shown above, right after contact import. What's even funnier, it lost all contacts afterwards - all names set to "Unknown contact".
So my "fragile" remark.
Any ideas?
Also, base _looks_ perfectly right, until I try to use autobackup. I can compact it, I can export settings, I can check history of any random contact. But not backup it.
-
Also, base _looks_ perfectly right
Yes, but base was broken in 0.95.8 (libmdbx 0.9). It worked, but it was already damaged inside. "Living dead". Autobackuper "shakes up" database completely and the problem is revealed
Ticket created about Import problem.
https://github.com/miranda-ng/miranda-ng/issues/1586
@ghazan will look at it in a week
-
Thank you. I will be waiting.
-
Any news?
Meanwhile I lost my USB-stick (third in month :|, and recovered DB is not accessible to Miranda. Old DB (only remained, as autosave stopped to work) was from 7th September, and as I just discovered, it missed about quarter of contact history. And when I tried to import data here from recovered DB, most of contacts was added again, effectively duplicating database. Ugh.
Question is, when and if recovery tools for new format will be available, and how can I import anything? :)
-
Smiling Spectre,
does Miranda exit after displaying this error box?
these tools are available, but the only thing they cannot do is to reimport data back from a text file
-
Smiling Spectre,
does Miranda exit after displaying this error box?
Yes, before (import from old "good" DB->new semi-broken DB) it exit just as I press "ok" in error box.
Now (new broken DB->old semi-broken DB from previous step) it recovered some of data, duplicated some more and ignored about 1/4 for reason unknown for me. No error anymore though. But last time both bases are after-crash.
Also, previous bug is in place: any import works only if I allow to import contacts.
these tools are available, but the only thing they cannot do is to reimport data back from a text file
I meant cleanup/recovery tools that was integrated in old format Miranda. Like "remove broken chains, clean up trash and duplicates, fix structure". It would fix any problems in old structure, and now, well, I have three bases, and all trhee are broken, cannot be repaired individually, and I even cannot merge it's data properly, it seems, least fix everything.
Thank you for your answer anyway. At least I know that someone heard of the problem. :)
-
I think you should
1) create the clean new database (via Miranda NG 0.95.9) and import one of the old databases (contacts, history, etc).
2 сreate another clean new database and import another old database.
3) сreate another clean new database and import third old database.
Do not try to import one corrupted db to another corrupted db.
After that, we can start to think about how to combine this 3 new databases.
But you still may encounter "Assertion failed" problem... so see next post.
-
Another solution (if Miranda 0.95.9 can start with "broken" databases):
1) run Miranda NG 0.95.9 with first "broken" database, install Msg_Export plugin and export history per-contact to JSON format. Or install History++ or mContacts plugin and export history per-contact to DAT format. Repeat this for every contact.
2) Repeat this for db #2 and db #3
3) Create new clean db in Miranda NG 0.95.9 and import only contacts from "broken" database (because "any import works only if I allow to import contacts")
4) Import history files from steps 1 and 2 via contact menu → import... → check "check duplicates" option and pick up corresponding .json or .dat. Repeat for every contact.
-
Another solution (if Miranda 0.95.9 can start with "broken" databases):
1) run Miranda NG 0.95.9 with first "broken" database, install Msg_Export plugin and export history per-contact to JSON format. Or install History++ or mContacts plugin and export history per-contact to DAT format. Repeat this for every contact.
2) Repeat this for db #2 and db #3
3) Create new clean db in Miranda NG 0.95.9 and import only contacts from "broken" database (because "any import works only if I allow to import contacts")
4) Import history files from steps 1 and 2 via contact menu → import... → check "check duplicates" option and pick up corresponding .json or .dat. Repeat for every contact.
Sounds like a good plan for me. Thank you. I'll try your first method, and if it will not work... *sigh* I'll consider second. Exporting 300 contacts trice, then importing it sounds like a chore, but there is no export with more convenient way, right?
-
I do not know how to export the history of all contacts... maybe MirLua can automate this, but I do not know how to write Lua (https://en.wikipedia.org/wiki/Lua_(programming_language)) scripts.
-
MsgExport natively supports exporting history of all contacts into separate files (into Jsons if you have the recent versions), so it's just one click
-
MsgExport natively supports exporting history of all contacts into separate files (into Jsons if you have the recent versions), so it's just one click
Can you explain how to do that, please? For me it simply move old .txt files, if they existed on old place (and they existed since 2014 at least), or do nothing if I remove all this files.
-
If you install MsgExport, then go into miranda settings->History->Message Export, you can (on the contacts tab) highlight all the contacts whose history you want to export. Not only will this apply to when they speak, but there is a button called "Export all history" which - if pressed - exports all the history of the contacts whose name has been selected - into txt files. Make sure to have a valid path selected in the general options of the plugin, and make sure to have the "json format for export" selected to have it work with recent versions of import later on.
I can post a screencap of my settings if you wish, but this should be good.
-
If you install MsgExport, then go into miranda settings->History->Message Export, you can (on the contacts tab)
Ah! It's _there_! Thank you, found it.
Broken with the same "Assertion failed" though. It seems, some contacts are broken... Well, at least I can try to experiment with it.
---------
Later
---------
Yes, it's per contact, and I can see the place of error in History++ (Message export keep to export data just to this place, then broke). Theoretically speaking, I can circumvent it somehow...
...But there is other problem appeared: if I have the same contact in the several protocols, it exports everything into the same one file, with header of one, first protocol. I can circumvent it by renaming contacts manually - but I think it worth to fix it in future plugin releases. :)
-
Actually the plugin is fairly sophisticated and indispensable. It already offers this functionality - you can specify it using the variables.
(https://i.imgur.com/zSfLhgt.png)
-
Actually the plugin is fairly sophisticated and indispensable. It already offers this functionality - you can specify it using the variables.
(https://i.imgur.com/zSfLhgt.png)
My fault. :( I totally could find it myself.
Thank you for explanation!
Just in case I missed something else: what "clear" in context menu means? And is there way to un-tick everything? (So far I use mouse selection - export, but with ticks it would be slightly more transparent... if I could select needed contacts between three hundreds).
-
Also you can try our new mdbx_dump and mdbx_load tools
Just update to development version, open command promt in Miranda folder and use:
mdbx_dump.exe -n -a -f 1.tmp "path/to/corrupted.dat"
mdbx_load.exe -n -f 1.tmp repaired.dat
del 1.tmp
This extract as much data as possible from the corrupted.dat to text file and build repaired.dat
-
I will try that, thank you!
-
So today miranda randomly corrupted my DB like has been the case lately, so I had a chance to test these tools. I noticed my DB had been corrupted when I typed something in a discord chat and it did not appear in the chat window despite being sent.
-> mdbx_dump.exe: D:\Temp\Vulpix.dat: MDBX_CORRUPTED: Database is corrupted
Not sure if it knows the db is corrupted or if it says this by default; I believe we are using mdbx sort of against its spec or something, so it'll always think it's corrupted. Maybe @Ghazan can comment.
-> mdbx_load.exe: took incredibly long to run. I guess about 20 minutes on a 473MB DB (well it was ~680MB when "dumped") with somewhat high disk activity. Get your tea and sit back for this one.
Recovered DB -> on startup, appears as completely empty, some settings lost. No accounts or protocols or users. History++ shows "ACCOUNT UNLOADED" for every contact, but contacts are present there. Still..
Trying to import this DB instead of using the recovery tool, it shows "Skipping contacts with no protocol" and imports nothing.
Instead, trying to import the corrupted db with miranda by creating clean db and importing it via import.dll produced better results, actually preserving all of the data up till when the failure occurred apparently. Well, not all. It deleted "duplicates" which were not duplicates but events with identical second timestamps. So data is lost in this case. @ghazan can you please disable this checking on db import?
And well for the mdbx tool introduced, what is the use case of the recovery tool?
Also @unsane, how is the sqlite db coming along? :p I'll host it on raid0'd SSD if it's stable and simply slow.
Oh also the best thing ever. The repaired profile actually loaded some default settings I guess, and what that did is it deleted all of the old backups in the autobackup folder :D so now I have no backups other than exported jsons + from past week when I do my actual offline backups.
I can recover everything but damn. This fell through even my expectations.
-
Smiling Spectre,
the original problem with
=========== 8< =========== >8 ===========
---------------------------
Microsoft Visual C++ Runtime Library
---------------------------
Assertion failed!
=========== 8< =========== >8 ===========
should have been fixed in the development version. would you please check it?
-
Sorry, I am slightly distracted by other activities, but I'll try to check it somewhere at weekend.
-
And sorry again, there was too many different events that delayed everything. :(
Ok, so results are:
1. Import now works! Too bad that it excluded mostly all my newer database history. :'( Seems, broken history is lost forever. But old one, that was broken before, loaded just right. Thank you!
2. After I made sequential import from march to december, I got mostly everything back. Except gaps here and there where base went crazy and lost history. :| Well, will do.
3. New fix and export tools doesn't work, and I have no idea why. It gives me errors, but it looks like Russian in bad codepage. See attachments.
Otherwise, it works at last. Thank you! :)