Editing User:Chriswaterguy/Tech/Importing templates with dependencies

Note, regex{{w|regular expressions}} skills needed for the xml editing. 
# Export from source wiki, turning on "Include templates." '''Do not import this file.''' 
# Determine the list of pages to be imported. 
## Delete everything except the title lines and turn those titles into wikilinks. (One method: Remove all linebreaks - regex matching '''''\r?\n''''' should work on most systems - then regex replace '''''<nowiki>.*?<title>([^<]+)</title></nowiki>''''' with '''''<nowiki>\* [[$1]]\n</nowiki>''''' and delete the last paragraph, which will not be a page title.)
## Paste into an Appropedia page (e.g. the [https://www.appropedia.org/w/index.php?title=Appropedia:Sandbox&action=edit&section=new Sandbox]). 
## Delete all the blue links from the list, until that only redlinks remain. Be 100% sure, to avoid overwrite problems. (This assumes that the blue links represent the same template working close enough to the same way on both wikis.)
## For each template in the list, add a /doc page below it. Replace '''''<nowiki>(\* \[\[Template\:[^]]+)\]\]</nowiki>''''' with '''''<nowiki>$1]]\n$1/doc]]</nowiki>'''''
## ''On second thoughts, this step may not be necessary as importing a non-existent page should simply fail with no action taken.'' <tt>Take this new list and paste it into [https://en.wikipedia.org/w/index.php?title=Wikipedia:Sandbox&action=submit Wikipedia's Sandbox]. Click "Preview". Remove any redlinks. Preview to check. This is the list to be exported from the source wiki.</tt>
# Go to the export page on the source wiki
## Paste in the short list of red links (removing the square brackets)
## '''Turn off "Include templates"'''
## Turn on "Save as file"
## Export 
# Edit the xml export file: 
## Strip out interlanguage links, if any - step through each one to avoid false positives. This regex match code should get almost all, with few false positives: '''''^\[\[([a-z][a-z][a-z]?[\:\-]|simple\:)[^\]]+\]\](\r?\n)?''''' and hit delete or replace with nothing.
## Change every username - for Wikipedia, assuming you've imported only the latest revision, change usernames to "(various editors @ en.wikipedia.org)". Regex: Match '''''<nowiki>\<username\>[^\<]+\</username\></nowiki>''''' and replace with '''''<nowiki><username>(various editors @ en.wikipedia.org)</username></nowiki>'''''
# Double check there are no existing templates (e.g. do a quick search and eyeball each <title> line, comparing it to your memory - what you expected to be there.)
# Import here.
# Check the templates.
# Add attribution, e.g. by pasting in '''''<nowiki>{{attrib wikipedia|{{FULLPAGENAME}} }}</nowiki>''''' at the bottom of the "noinclude" section. (This code assumes the template is from Wikipedia and the name is unchanged.)


== Issues ==

* UTF-8 errors: Failed to import some files (I deleted beginning of xml file and started again, but these files were still blank). These have been manually imported:`
** [[Template:Category handler/blacklist/doc]]{{w|Template:Category handler/blacklist/doc}}
** [[Template:Category handler/numbered/doc]]{{w|Template:Category handler/numbered/doc}}
** [[Template:Category handler/doc]]{{w|Template:Category handler/doc}}
** [[Template:Fmbox/doc]]{{w|Template:Fmbox/doc}}
** [[Template:High-risk/doc]]{{w|Template:High-risk/doc}}
** [[Template:Intricate template/doc]]{{w|Template:Intricate template/doc}}
** [[Template:Lts//doc]]{{w|Template:Lts//doc}}
** [[Template:Ns has subpages/doc]]{{w|Template:Ns has subpages/doc}}
** [[Template:Pp-meta/doc]]{{w|Template:Pp-meta/doc}}
** [[Template:Sometimes/doc]]{{w|Template:Sometimes/doc}}
* [[Template:Lts//doc]] seems to silently redirect to [[Template:Lts/doc]]. History page is ok, via url hack. Doesn't transclude to [[Template:Lts/]] either - but it works fine in Wikipedia, => see if it's fixed with a MediaWiki upgrade?

== NEXT STEPS==
Late 2012:
* Add '''<nowiki>{{attrib wikipedia|{{subst:PAGENAME}}}}</nowiki>''' to each imported template. (On template page before last noinclude, for greatest reliability... Or on the /doc page so it's on every relevant page, including the /doc page when viewed directly. A bit fiddly to deal with the cases of no /doc, but there are very few such cases.)
* [[Template:Sometimes]]{{w|Template:Sometimes}} looks wrong

Weird addition of content (early revisions of mainspace content) to failed templates, e.g. STAR-TIDES page at [[Template:Sometimes/doc]].

== Next templates to import ==

Do next mass import after server upgrade to avoid charset errors, and preferably when we have enabled either multiple image upload, or displaying images from Commons.

* [[Template:Doi]] - copy again, template plus subpages.
* [[Template:To do]]
* [[Template:More information]]
* [[Template:Details]]
* [[Template:Hatnote templates documentation]]
* [[Template:Superimpose]]
* [[Template:Attribution needed]]
* [[Template:By whom]]
* [[Template:According to whom]]
* [[Template:Weasel-inline]]
* [[Template:Which]]
* [[Template:Who]]
* [[Template:Better source]]
* [[Template:Citation needed]]
* [[Template:Citation needed span]]
* [[Template:Citation needed (lead)]]
* [[Template:Failed verification]]
* [[Template:Full]]
* [[Template:Primary source-inline]]
* [[Template:Request quotation]]
* [[Template:Retracted]]
* [[Template:Third-party-inline]]
* [[Template:Author missing]]
* [[Template:Author incomplete]]
* [[Template:Date missing]]
* [[Template:ISBN missing]]
* [[Template:Page needed]]
* [[Template:Publisher missing]]
* [[Template:Title incomplete]]
* [[Template:Year missing]]
* [[Template:Contradict-inline]]
* [[Template:Contradiction-inline]]
* [[Template:Disputed-inline]]
* [[Template:Dubious]]
* [[Template:Examples]]
* [[Template:Inconsistent]]
* [[Template:List fact]]
* [[Template:Lopsided]]
* [[Template:MIR]]
* [[Template:POV-statement]]
* [[Template:Clarify timeframe]]
* [[Template:Quantify]]
* [[Template:Update after]]
* [[Template:Update-small]]
* [[Template:When]]
* [[Template:Where]]
* [[Template:Year needed]]
* [[Template:Disambiguation needed]]
* [[Template:Pronunciation needed]]
* [[Template:Ambiguous]]
* [[Template:Awkward]]
* [[Template:Buzz]]
* [[Template:Clarify span]]
* [[Template:Definition]]
* [[Template:Elucidate]]
* [[Template:Expand acronym]]
* [[Template:Technical-statement]]
* [[Template:Vague]]
* [[Template:Why]]
* [[Template:Other uses-section]]
* [[Template:For]]
* [[Template:Distinguish]]
* [[Template:User time zone]]
* [[Template:Cite quote]]
* [[Template:Request quote]]
* [[Template:Verify source]]
* [[Template:Verify credibility]]
* [[Template:Citecheck]]
* [[Template:Not verified]]
* [[Template:Unreferenced]]
* [[Template:page number]]
* [[Template:dubious]]
* [[Template:Who]]
* [[Template:Date]]

+ask Teratornis.

Categories/groups of templates (import category pages, but also need to add each template, I think):
* anything asking to expand or add references/sources
* [[wikipedia:Category:Creative Commons copyright templates]]
* some or most of: [[wikipedia:Category:Wikipedia_image_copyright_templates]]

Import then rename:
* [[Template:WPBannerMeta]]   ->ColabMeta? Approcolab meta?

Update, & edit doc page after import:
* [[Template:Clarify]] - use {{tl|verification needed}}  ({{tl|specify}} is too vague and seems redundant)

Tools or processes to import and/or adapt, with their templates. (This will take work to identify & list the templates.):
* [[wikipedia:WP:Article wizard]] - maybe simplify the steps by using a semantic form. Work with someone familiar with the Wikipedia process.
** See also [[wikipedia:WP:Articles for creation]] (for anons) and [[wikipedia:WP:Article Incubator]] (a soft deletion process, which is also one role for our [[A:Incubator]]. I think it will be easiest to keep all unready articles under one parent page, such as '''Appropedia:Incubater/...''').
* [[wikipedia:WP:Article Rescue Squadron]] - way to big for us to manage right now, but might have some useful ideas and templates. A project for a helpful wikiholic Wikipedian? 
* [[wikipedia:Template:Articles by Priority]] & maybe other from [[wikipedia:Category:Classification templates]].

Manual import & edit:
* Think about adapting [[wikipedia:Template:WikiProject Norway]] (or any such talk page notice) - worth trying one or two such groups on Appropedia? As long as it's not relying just on conventional talk pages for communication - need some combination of a bot to update talk pages (opt-in), alerts for changes to that page (separate from standard watchlist alerts), an active facilitator, a related academic program working on the area, and/or a social media person promoting the area and the collaboration.