Database migration

[riki]

Many archiving programs, such as RAR, could open write-locked files.
But keep in mind that such method is prone to making an inconsistent backup.

That's why I asked if it's possible to flush the database file and leave it in a consistent state, just for the sake of backup, without stopping Miranda.

Your best option is something like https://wiki.miranda-ng.org/index.php?title=Plugin:Db_autobackups
Haven't used it myself, no idea if it's working with current database driver.

Good advice, AnrDaemon. I'll install this plugin and see. The download page says it should work with the mdbx database driver. However, there's a warning that enabling the "zip it" option might produce corrupted backups.

EDIT 1 - I had to choose the experimental version (the stable version didn't appear in plugin list). It appears to work fine and creates neatly named backups. Haven't verified if the backup files are consistent, though.

EDIT 2 - No, the plugin can't actually copy the database. The backup file gets opened/created, then the copy aborts and the backup gets closed/deleted. I guess it's the same locked profile.dat problem. Nice try anyway 

[dartraiden]

Db_autobackups works perfectly for me.

Can you provide Version Info report and maybe some important info? (for example, "profile located on network drive", "profile path contains non-ascii symbols", etc)

Zip-packing is broken in some circumstances (it produces corrupted archives on my virtual machine with Cyrillic paths and long paths).

[riki]

Thanks for your interest, dartraiden.

Here's my version info.

https://pastebin.com/gQRQcqUV

There are no special mentions to be made: no non-ISO-Latin characters in pathname, save path for backups is
C:\Documents and Settings\TheUser\Application Data\Miranda\Profiles\default\AutoBackups\
as suggested by the plugin.

What happens is the file gets created (it appears in the window for a short time), then it disappears. Actually, it is the same behavior I observe when I try to manually make a copy in the same directory as profile.dat, and I guess is the same problem that rsync finds when it tries to copy the profile to my remote rsync server.

EDIT - In my version info, the autobackup plugin is disabled. I turned it off myself because it ddn't work after extensive testing. I'll turn it back on now, just to able to report in case of questions from you or some other helping soul.

[dartraiden]

Profile: C:\Documents and Settings\TheUser\Application Data\Miranda\Profiles\default\profile.dat (MDBX database driver)

 

Miranda even shouldn't start if bolded parts differ. This parts MUST be identical.

[Goraf]

From Edit 1 and previous post seems that you all the time trying to mix up things. Like it was mentioned before to you, it is very important to update plugins only by PluginUpdater. By not doing that you are asking yourself for unpredictable problems. The same applies for installing them. For that purpose you have "Available component list" entry in main menu. Please remember about that in the future

[dartraiden]

I had to choose the experimental version (the stable version didn't appear in plugin list)

Just tested on stable

[riki]

Oops!
Sorry guys, thanks for taking the time for setting me straight. What actually happened is I'd edited the version info to keep my full name from appearing (profile at the moment is named firstname-lastname.dat).

I'd installed the autobackup plugin manually (no available component list, ehm), but I think the installation was correct. The package contained only the dll file. But I only update via updater, always.

Now I removed the autobackup plugin and used "Available Component List" to install it. No choice stable/experimental, maybe that just happened because I went straight to the plugin download page.

Here's the new version info - only last 2 components of profile.dat are slightly edited. Now I also removed the unloadable plugins, the folder is clear of rubbish.

https://pastebin.com/6MY0JSF4

P.S. I will remember to use only "Available Component List" in the future. Thank you.

[dartraiden]

No choice stable/experimental

Updater always shows plugins list according to Miranda version (stable plugins for stable version, etc).

[dartraiden]

Please test your backup utility (rsync?) after next development version update.
https://github.com/miranda-ng/miranda-ng/commit/89ac15679e32fa707757a95fafbfc7ef9b3e705d

Another way: use backup tool which supports shadow copies

[riki]

Thanks for bearing with me, dartraiden.

I've updated Miranda twice since my last message. My versioninfo is here: https://pastebin.com/6mUkUzh6

I've visited the github page you linked, and it seems like it should solve my problem, but unfortunately neither rsync nor Miranda's autobackup plugin are yet able to copy the file while Miranda is running.

Your suggestion about shadow copies is a good one! Setting up the shadow copy is a bit of a pain, as it requires an extra script etc., but I already have the necessary machinery in place to make backups of the registry about once-twice a month. So maybe I should create a new daily backup routine just for my profile.dat, with all the shadow copy mechanism in place. I'll have it launch at a different time from the "normal" daily backup (no shadow copy necessary there), as to avoid conflicts or excess disk stress.

EDIT
After last update (2018-04-13, db drivers and more stuff), the db file still appears to be locked - so no live running backup is possible either by autobackup plugin or rsync without tinkering with shadow copies.

However, RAM use by Miranda has dropped by 30-35%. Wow! 

[Vulpix]

Most of various db migration issues have been resolved in the past few days thanks to Ghazan; people who have been holding off on migration, please make a backup and try it, see how it works for you. All good over here!

[sir_qwerty]

Updater always shows plugins list according to Miranda version (stable plugins for stable version, etc).

The same assumption exactly sent me into the endless loop when updating database and migrating to new db driver in alpha build. I experienced this scenario:

• Initially I was on stable build 0.95.7 with updater set to check for stable plugins
• I switched manually to alpha 0.95.8 build which resulted in db conversion
• At this point I had no idea there is any setting in pluginupdater hadn't changed it so it was instructed to stick with stable builds
• PluginUpdater found 0.95.7 and replaced manually installed alpha with it
• This reverted whole installation to old stable release and rendered database unusable by 0.95.7 upon restart
• It took me like one hour to get out of this carousel and to set PluginUpdater to stick with alpha builds

So it seems to me that PluginUpdater doesn't stick with Files source according to core version but according to the setting.

[Goraf]

Updater always shows plugins list according to Miranda version (stable plugins for stable version, etc).

Yes, but version you choose. So it should be understood as "according to Miranda version you have, and that version is the one you choose in options"

That's why we have things like (see screenshot) or similar in many places.
That's why we have https://wiki.miranda-ng.org/index.php?title=Plugin:PluginUpdater#Version_choice or https://wiki.miranda-ng.org/index.php?title=Installation_and_update#Updating_Miranda_NG

What do you think we should do more or where when your answer suggest that it is still not 100% visible or clear? We are open on suggestions or advice in this manner.

[riki]

Hello, trouble here. Miranda starts with an error message.

Miranda32.exe has encountered a problem and needs to close...

For more information about this error:

AppName: miranda32.exe
AppVer: 0.95.8.19810
ModName: dbx_mdbx.dll

To view technical information about this error report, click here:
(Can't copy/paste the ensuing dialog box. Suggestions?)

Additional information in \Documents and Settings\...\224f_appcompat.txt
I pasted here: https://pastebin.com/SR591yym

If I try to use the commandline MimCmd.exe, I get

 
Could not create connection with Miranda or could not retrieve list of known com
mands.

I think I lost my database again.  Last backup is 3 days old, since I backed up with Miranda stopped, or the file wouldn't copy.

EDIT - It seems that database file size has increased significantly since the update, although my Miranda activity is average as usual.

Last backup before migration (but I think I was already on new driver) - 183,697,408 bytes
First backup after migration (this database won't open in Miranda, it got damaged I think) - 268,435,456 bytes
Last backup after migration (3 days old, opens OK) - 536,870,912 bytes

I also find memory (RAM) use by Miranda to oscillate wildly between ~70 MB and ~600 MB over the hours/half hours.

Is this expected?

(I hope I'm not a nuisance and my reports help improve Miranda)

[Vulpix]

For database size increasing post migration, this is expected; since we now store indices as well (no longer a linked list like mmap was).

For example my database used to be about 297 MB (properly maintained with no wasted space); post migration it is now 420MB. But it shouldn't randomly, wildly increase afterwards.. it should grow at about a normal pace.