Appropedia talk:Current dump
Add topicScript to include date in the dump file filename
[edit source]The date of the latest dump could also be put in the filename, which would be handy... but we're not quite sure how to do this. If you like this idea, and know exactly how to do it, please leave a note on the talk page - thanks!
Going by the script posted in the page, the following modification would do the trick on a POSIX-compilant system:
cd /home/apropos/appropedia.org/approbackups /usr/local/php5/bin/php ../maintenance/dumpBackup.php --current | gzip > appropedia-$(date +%F)-dump.xml.gz
Note the added date command. It should produce filenames like this:
appropedia-2012-03-20-dump.xml.gz
Also note that I'm using the ISO date format. Among other things, it have the benefit that a computer could sort them using simple alphabetical ordering. (That could be useful if someone plans to store more than one version of the dump.)
--Dragonfi 14:17, 20 March 2012 (UTC)
- Thank you! I've added it to our tech workstream. When we have a volunteer or intern to work on tech things (hopefully soon!) we'll look at getting this implemented. --Chriswaterguy 05:16, 21 March 2012 (UTC)
proposition to share backups of media files
[edit source]May be that could be a big compressed tar file that could be shared on another server ? maybe rsync could be used to transfer media files to the other server so that there won't be too much bandwidth used on the main server ? -- Pierro78 (talk) 01:36, 16 November 2016 (UTC)
no data in dump file
[edit source]As of 14 August 2017, the last two (and maybe more) dump files have been completely empty - a 0 byte XML file inside of a 20 byte gzip archive. Can somebody look into this? STILL BROKEN as of 2017-11-16
- We (appropedia) would love assistance on this. Thanks! --Lonny (talk) 15:02, 18 November 2017 (UTC)
- Hi folks, I realize it's been a while, but would you still like help getting dumps working again? Dethe (talk) 06:15, 16 February 2025 (UTC)
- @Dethe Hi, yes! I'm Felipe, the current lead developer at Appropedia. We'd honored to get your help in getting the dumps going again. However, when the dumps worked I still hadn't joined Appropedia, so I don't know of how it was set up, or if you were helping back then or not. Could you tell me about you, you story with Appropedia, and current motivations? I'd be very much interested in knowing! Kind regards, Sophivorus (talk) 16:17, 18 February 2025 (UTC)
- Hi Filipe, good to (virtually) meet you.
- My story with Appropedia is just that I have known of it for years, see value in it, and see the need for it only increasing as the world progresses down the path we're on. My motivation is fairly selfish: I'd like to have a local copy and be able to distribute Appropedia and other key resources by sneakernet in case of worst case scenarios. I'm also more or less retired, so I can put some time into it.
- I don't have a lot of experience with PHP, but I do have 30+ years programming experience. Most recently it's been primarily JavaScript and Python, but probably 15 languages or so over the years. I don't know how the data dumps were set up before, but I have had to do software archeology before and I'm intimately familiar with XML and walking trees (like you would do for say, scraping a site).
- I hope that helps! Happy to answer any further questions you have. Dethe (talk) 17:29, 18 February 2025 (UTC)
- @Dethe Hi, good news! While researching my reply to you, I found dumpBackup.php, a standard MediaWiki maintenance script to generate XML dumps, and it was so straightforward that I went ahead and set up a monthly cronjob to produce two dumps, one with files, one without. See Appropedia:Dumps for the details and let me know if these dumps serve your needs or if you need something else (for example, a dump including the history of the pages, or some namespace other than the main namespace). It would be an honor to have your help on this one, but the script seems so straightforward that I wouldn't want to waste your time. That being said, if you'd still like to help us expand or refine the dumps, or if we can help you in any other way, just let me know! Sophivorus (talk) 15:36, 21 February 2025 (UTC)
- @Dethe Hi, yes! I'm Felipe, the current lead developer at Appropedia. We'd honored to get your help in getting the dumps going again. However, when the dumps worked I still hadn't joined Appropedia, so I don't know of how it was set up, or if you were helping back then or not. Could you tell me about you, you story with Appropedia, and current motivations? I'd be very much interested in knowing! Kind regards, Sophivorus (talk) 16:17, 18 February 2025 (UTC)