Migration to Org mode
Published on Aug 18, 2019
Transform Markdown Files into Org files
Using bash script and pandoc to do this transform.
for f in `ls *.md`; do echo "[*] process: $f" pandoc -f markdown -t org -o ${f%.md}.org ${f}; done
however, some error happened.
➜ blog for f in `ls *.md`; do
for> echo "[*] process: $f"
for> pandoc -f markdown -t org -o ${f%.md}.org ${f};
for> done
[*] process: 2012-02-07-vim-latexxelatex.md
[*] process: 2012-02-23-vimlisp.md
[*] process: 2012-03-06-vbox.md
[*] process: 2012-03-11-vrdp.md
[*] process: 2012-03-21-lfs.md
[*] process: 2012-03-25-archlinux3264.md
[*] process: 2012-04-13-How-I-built-this-blog.md
[*] process: 2012-04-16-half-hackerhalf-artist.md
[*] process: 2012-04-24-.md
[*] process: 2012-04-26-a-text-game-engine-written-by-lisp.md
[*] process: 2012-04-28-visualize-tree-like-data-in-lisp.md
[*] process: 2012-05-01-grand-theft-wumpus.md
[*] process: 2012-05-03-the-orc-battle-game.md
[*] process: 2012-05-06-using-loop-to-evolve.md
[*] process: 2012-05-09-attack-of-the-robots.md
[*] process: 2012-05-10-streams-in-lisp.md
[*] process: 2012-05-12-web-server-written-in-lisp.md
[*] process: 2012-05-14-.md
[*] process: 2012-05-18-.md
[*] process: 2012-05-20-.md
[*] process: 2012-05-23-dice-of-doom.md
[*] process: 2012-05-26-.md
[*] process: 2012-05-28-archlinuxpdf.md
[*] process: 2012-05-31-tail-call-optimization.md
[*] process: 2012-06-02-memoization-in-functional-style.md
[*] process: 2012-06-05-scheme.md
[*] process: 2012-06-08-first-ai-in-lisp.md
[*] process: 2012-06-09-gimp.md
[*] process: 2012-06-12-.md
[*] process: 2012-06-13-macro-in-lisp.md
[*] process: 2012-06-16-dsl.md
[*] process: 2012-06-19-.md
[*] process: 2012-06-21-linux.md
[*] process: 2012-06-22-a-mathjax-test.md
[*] process: 2012-06-23-.md
[*] process: 2012-06-26-common-lisp.md
[*] process: 2012-06-28-google-glasses.md
[*] process: 2012-06-29-dice-of-doom-v2.md
[*] process: 2012-07-02-webdice-of-doom.md
[*] process: 2012-07-05-.md
[*] process: 2012-07-07-dice-of-doom-v4.md
[*] process: 2012-07-09-land-of-lisp.md
[*] process: 2012-07-11-.md
[*] process: 2012-07-13-.md
[*] process: 2012-08-04-.md
[*] process: 2012-08-11-.md
[*] process: 2012-08-15-galgame.md
[*] process: 2012-08-17-to-be-an-epicurean.md
[*] process: 2012-08-22-numpy.md
[*] process: 2012-08-24-scipy.md
[*] process: 2012-08-30-sympy-tutorial.md
[*] process: 2012-09-07-matplotlib-tutorial.md
[*] process: 2012-09-12-.md
[*] process: 2012-09-20-.md
[*] process: 2012-10-14-always-look-on-the-bright-side-of-life.md
[*] process: 2012-10-14-scikit-learn-machine-learning-in-python.md
[pandoc warning] Could not parse YAML header: did not find expected comment
or line break "source" (line 41, column 3)
[*] process: 2012-10-19-python.md
[*] process: 2012-10-22-scipy.md
[*] process: 2012-10-30-.md
[*] process: 2012-10-31-python.md
[*] process: 2012-11-12-numpyscipy.md
[*] process: 2012-11-19-flask.md
[*] process: 2012-12-02-python.md
[*] process: 2013-01-06-summary-and-prospect-in-2012.md
[*] process: 2013-01-13-moving-to-gentoo.md
[*] process: 2013-01-26-data-analysis-with-open-source-tools.md
[*] process: 2013-01-28-visualize-your-shell-history.md
[*] process: 2013-02-05-visualize-the-friendship-of-renren.md
[pandoc warning] Could not parse YAML header: did not find expected alphabet
ic or numeric character "source" (line 15, column 2)
[*] process: 2013-02-19-yet-another-photomosaic-generator.md
[*] process: 2013-02-25-generate-ascii-images-like-the-matrix.md
[*] process: 2013-03-02-python.md
[*] process: 2013-03-07-image-to-css.md
[*] process: 2013-03-11-quickstart-ebuild-guide.md
[*] process: 2013-03-13-advanced-python-constructs.md
[pandoc warning] Could not parse YAML header: did not find expected alphabet
ic or numeric character "source" (line 27, column 2)
[*] process: 2013-03-24-.md
[*] process: 2013-04-07-2-d-phase-unwrapping-path-following.md
[*] process: 2013-04-12-hello-bupt.md
[*] process: 2013-04-16-interfacing-with-c.md
[*] process: 2013-04-27-a-beautiful-night-in-jnu.md
[*] process: 2013-05-13-graduation-season.md
[*] process: 2013-05-16-some-books.md
[*] process: 2013-06-01-recent-summary.md
[*] process: 2013-06-09-hacking-the-art-of-exploitation.md
[*] process: 2013-07-02-goodbye-jnu.md
[*] process: 2013-07-22-build-up-your-own-pt-lab.md
[*] process: 2013-07-26-shellcode-black-magic.md
[*] process: 2013-08-03-heap-unlinking.md
[*] process: 2013-08-13-notes-on-anti-virus-evasion.md
[*] process: 2013-08-21-new-travel.md
[*] process: 2013-08-26-win32-shellcode.md
[*] process: 2013-08-31-so-boring.md
[*] process: 2013-09-06-.md
[*] process: 2013-09-09-.md
[*] process: 2013-09-13-.md
[*] process: 2013-09-15-.md
[*] process: 2013-09-30-.md
[*] process: 2013-10-11-.md
[*] process: 2013-10-15-wheres-the-way.md
[*] process: 2013-10-31-feelings-up-and-down.md
[*] process: 2013-11-09-what-a-fucking-life.md
[*] process: 2013-11-25-cloud-atlas.md
[*] process: 2013-12-21-.md
[*] process: 2013-12-24-writing-drivers-in-linuxa-brief-tutorial.md
[*] process: 2013-12-31-.md
[*] process: 2014-01-11-thanks.md
[*] process: 2014-01-12-python.md
[*] process: 2014-01-29-python.md
[*] process: 2014-02-18-.md
[*] process: 2014-02-26-spoofing.md
[*] process: 2014-03-04-play-arp.md
[*] process: 2014-03-07-play-dhcp.md
[*] process: 2014-03-15-play-icmp.md
[*] process: 2014-04-03-gns3-on-gentoo-linux.md
[*] process: 2014-04-18-.md
[*] process: 2014-05-04-360-hack-game.md
[*] process: 2014-05-04-.md
[*] process: 2014-05-10-gentoolaunchpad-msp430.md
[*] process: 2014-05-18-a-glance-at-d3js.md
[*] process: 2014-05-31-mysql-learning-notes.md
[*] process: 2014-06-02-.md
[*] process: 2014-06-13-.md
[*] process: 2014-07-26-upgrade-gentoo-linux.md
[*] process: 2014-08-07-radare2-shellcode.md
[*] process: 2014-08-08-better-and-beautiful-console.md
[*] process: 2014-08-08-defeating-ioli-with-radare2.md
[*] process: 2014-08-14-fbi-tor-malware-analysis.md
[*] process: 2014-08-26-.md
[*] process: 2014-09-08-escape-plan.md
[*] process: 2014-09-21-quora.md
[*] process: 2014-10-09-lrc.md
[*] process: 2014-10-09-.md
[*] process: 2014-10-11-gcjust-for-fun.md
[*] process: 2014-10-13-on-disk-encryption.md
[*] process: 2014-10-15-dwarf-fortress.md
[*] process: 2014-10-17-acl-on-linux.md
[*] process: 2014-11-12-some-articles.md
[*] process: 2014-11-15-.md
[*] process: 2014-11-17-linux-audit.md
[*] process: 2014-11-18-file-permissions-and-masks.md
[*] process: 2014-11-20-account-and-access-control.md
[*] process: 2014-12-24-.md
[*] process: 2015-03-02-2015s-first-post.md
[*] process: 2015-03-09-.md
[*] process: 2015-03-11-shellcodeenvironment.md
[*] process: 2015-03-16-shellcodeloaders.md
[*] process: 2015-03-16-shellcode.md
[*] process: 2015-03-17-shellcodedynamic.md
[*] process: 2015-03-18-load-time-relocation-of-shared-libraries.md
[*] process: 2015-03-19-position-independent-code-pic-in-shared-libraries.md
[*] process: 2015-03-30-boot-sequence.md
[*] process: 2015-04-01-python-spider-quick-and-dirty-way.md
[*] process: 2015-04-02-baidu-music.md
[*] process: 2015-04-03-ydcx.md
[*] process: 2015-04-04-chaoxing.md
[*] process: 2015-04-05-yingxin.md
[*] process: 2015-04-07-python-spider-camera-find.md
[*] process: 2015-05-02-javascript.md
[*] process: 2015-05-10-.md
[*] process: 2015-05-17-17.md
[*] process: 2015-05-24-hacking-security.md
[*] process: 2015-06-01-security-in-browsers.md
[*] process: 2015-06-12-.md
[*] process: 2015-06-27-.md
[*] process: 2015-07-03-.md
[*] process: 2015-08-07-maybe-the-last-post-here.md
[*] process: 2015-11-12-i-am-back.md
[*] process: 2015-11-17-greasemonkey-userscript.md
[*] process: 2015-11-18-notes-on-migrating-from-jeykll-to-hexo.md
[*] process: 2015-11-21-the-es6-features-i-use-these-days.md
[*] process: 2015-11-23-linux-tex.md
[*] process: 2015-11-24-web-based-input-method.md
[*] process: 2015-11-29-using-buptgraduatethesis-on-ubuntu.md
[*] process: 2015-12-06-freedom.md
[*] process: 2015-12-14-wangyin.md
[*] process: 2015-12-20-stream-in-nodejs.md
[*] process: 2016-01-02-2015-summary-and-prospect.md
[*] process: 2016-01-15-webchat-user-bot.md
[*] process: 2016-02-09-play.md
[*] process: 2016-02-26-evolving-game-once-more.md
[*] process: 2016-02-28-say-something.md
[*] process: 2016-03-07-neural-network-101.md
[*] process: 2016-03-08-harvest-mm-pics.md
[*] process: 2016-03-09-black-hat-javascript.md
[*] process: 2016-03-22-yet-another-new-journey.md
[*] process: 2016-04-20-after-10-o-clock.md
[*] process: 2016-05-02-the-quieter-you-become-the-more-you-are-able-to-hear
.md
[*] process: 2016-05-25-for-freedom.md
[*] process: 2016-06-11-writing-something-useful.md
[*] process: 2016-06-26-goodbye.md
[*] process: 2016-07-24-exploring-webpack.md
[*] process: 2016-09-13-half-year-full-time-front-end-developer.md
[*] process: 2016-11-02-interview.md
[*] process: 2017-01-08-spring-and-chaos.md
[*] process: 2017-05-26-memorize-bbs-byr-cn.md
[*] process: 2017-06-14-ubuntu-gnome-guide-for-mbp-users.md
[*] process: 2017-06-17-develop-a-mind-which-does-not-abide-in-anything.md
[*] process: 2018-01-06-2018.md
[*] process: 2018-11-11-dwarf-fortress-story-calendar.md
[*] process: 2019-06-24-i-am-back-again.md
[*] process: 2019-08-11-eink-experience-in-two-weeks.md
➜ blog
The error occured.
process 2012-10-14-scikit-learn-machine-learning-in-python.md [pandoc warning] Could not parse YAML header: did not find expected comment or line break "source" (line 41, column 3)
obselete old toc implement when i use jekyll, which I write as
— *目录 {:toc} —
Insert Back FrontMatters
I use a python script to migrate jekyll yml frontmatter to orgmode options
FROM = "/home/reverland/blog-next/src/posts/" TO = "/home/reverland/tmpblog/" for f in os.listdir(FROM): if f.endswith(".md"): fm = frontmatter.load(os.path.join(FROM, f)) title = fm['title'] date = f[:10] summary = fm['excerpt'] tags = " ".join(fm['tags']) front = "#+TITLE: %s\n#+DATE: <%s>\n#+SUBTITLE: %s\n#+TAGS: %s\n\n" % (title, date, summary, tags) with open(os.path.join(TO, os.path.splitext(f)[0] + '.org')) as original: data = original.read() with open(os.path.join(TO, os.path.splitext(f)[0] + '.org'), 'w') as modified: modified.write(front + data)
Blog with Org mode
Org mode has a versatile export and publishing function.
I tweak a little from @narendraj9's org-blog.el.
(require 'org) (require 'ox-publish) (require 'ox-html) (require 'org-element) (require 'ox-rss) (defvar org-blog-date-format "%h %d, %Y" "Format for displaying publish dates.") (defun org-blog-prepare (project-plist) "With help from `https://github.com/howardabrams/dot-files'. Touch `index.org' to rebuilt it. Argument `PROJECT-PLIST' contains information about the current project." (let* ((base-directory (plist-get project-plist :base-directory)) (buffer (find-file-noselect (expand-file-name "index.org" base-directory) t))) (with-current-buffer buffer (set-buffer-modified-p t) (save-buffer 0)) (kill-buffer buffer))) (defvar org-blog-head "<link rel=\"stylesheet\" href=\"https://stackpath.bootstrapcdn.com/bootstrap/4.1.3/css/bootstrap.min.css\" integrity=\"sha384-MCw98/SFnGE8fJT3GXwEOngsV7Zt27NXFoaoApmYm81iuXoPkFOJwJ8ERdknLPMO\" crossorigin=\"anonymous\"> <link rel=\"stylesheet\" type=\"text/css\" href=\"/static/blog.css\"/> <link rel=\"stylesheet\" type=\"text/css\" href=\"https://fonts.googleapis.com/css?family=Amaranth|Handlee|Libre+Baskerville|Bree+Serif|Ubuntu+Mono|Pacifico&subset=latin,greek\"/> <link rel=\"shortcut icon\" type=\"image/x-icon\" href=\"/static/favicon.ico\">") (defun org-blog-preamble (plist) "Pre-amble for whole blog." ;; Hack! Put date for the post as the subtitle (when (s-starts-with-p "20" (plist-get plist :input-file)) (plist-put plist :subtitle (format "Published on %s" (org-export-get-date plist org-blog-date-format)))) ;; Return a simple banner with navigation links "<div class=\"banner\"> <a href=\"/\"> Reverland的行知阁 </a> </div> <ul class=\"banner-links\"> <li><a href=\"/archive.html\"> Posts </a> </li> <li><a href=\"/archive.xml\"> RSS </a> </li> <li><a href=\"/index.html\"> About Me </a> </li> <li><a href=\"/links.html\"> Links </a> </li> </ul> <hr>") (defun org-blog-postamble (plist) "Post-amble for whole blog." (concat "<footer class=\"footer\"> <a href=\"https://reverland.org\"> <p> Built with <svg id=\"i-heart\" viewBox=\"0 0 32 32\"> <path d=\"M4 16 C1 12 2 6 7 4 12 2 15 6 16 8 17 6 21 2 26 4 31 6 31 12 28 16 25 20 16 28 16 28 16 28 7 20 4 16 Z\"/> </svg> in <img id=\"i-emacs\" src=\"https://www.gnu.org/software/emacs/images/emacs.png\"/> <span id=\"view-source-link\"> copyleft@2012-2019 </span> </p> </a> </footer> <script type=\"text/javascript\" src=\"/static/blog.js\"> </script> <!-- Global site tag (gtag.js) - Google Analytics --> <script async src=\"https://www.googletagmanager.com/gtag/js?id=UA-38681562-1\"></script> <script> window.dataLayer = window.dataLayer || []; function gtag(){dataLayer.push(arguments);} gtag('js', new Date()); gtag('config', 'UA-38681562-1'); </script> " ;; Add Disqus if it's a post (when (s-contains-p "posts/" (plist-get plist :input-file)) "<!-- Disqua JS --> <div id=\"disqus_thread\"></div> <div id=\"disqus_thread\"></div> <script type=\"text/javascript\"> var disqus_shortname = 'reverlandblog'; (function() { var dsq = document.createElement('script'); dsq.type = 'text/javascript'; dsq.async = true; dsq.src = '//' + disqus_shortname + '.disqus.com/embed.js'; (document.getElementsByTagName('head')[0] || document.getElementsByTagName('body')[0]).appendChild(dsq); })(); </script> <noscript>Please enable JavaScript to view the <a href=\"http://disqus.com/?ref_noscript\">comments powered by Disqus.</a></noscript> "))) (defun org-blog-sitemap-format-entry (entry _style project) "Return string for each ENTRY in PROJECT." (when (s-starts-with-p "20" entry) (format "@@html:<span class=\"archive-item\"><span class=\"archive-date\">@@ %s @@html:</span>@@ [[file:%s][%s]] @@html:</span>@@" (format-time-string "%h %d, %Y" (org-publish-find-date entry project)) entry (org-publish-find-title entry project)))) (defun org-blog-sitemap-function (title list) "Return sitemap using TITLE and LIST returned by `org-blog-sitemap-format-entry'." (concat "#+TITLE: " title "\n\n" "\n#+begin_archive\n" (mapconcat (lambda (li) (format "@@html:<li>@@ %s @@html:</li>@@" (car li))) (seq-filter #'car (cdr list)) "\n") "\n#+end_archive\n")) (defun org-blog-publish-to-html (plist filename pub-dir) "Same as `org-html-publish-to-html' but modifies html before finishing." (let ((file-path (org-html-publish-to-html plist filename pub-dir))) (with-current-buffer (find-file-noselect file-path) (goto-char (point-min)) (search-forward "<body>") (insert (concat "\n<div class=\"content-wrapper container\">\n " " <div class=\"row\"> <div class=\"col\"> </div> " " <div class=\"col-sm-6 col-md-8\"> ")) (goto-char (point-max)) (search-backward "</body>") (insert "\n</div>\n<div class=\"col\"></div></div>\n</div>\n") (save-buffer 0) (kill-buffer)) file-path)) (use-package ox-publish :requires (org ox-html ox-rss) :commands publish-blog :config (defun publish-blog () (interactive) (org-publish-project "blog" t)) (setq org-confirm-babel-evaluate nil) (setq org-publish-project-alist `(("blog-content" :base-directory "~/org/posts/" :exclude ".*~" :base-extension "org" :publishing-directory "~/org/blog/" :recursive t :preparation-function org-blog-prepare :publishing-function org-blog-publish-to-html :with-toc nil :with-title org-export-with-title :with-date org-export-with-date :section-numbers nil :html-doctype "html5" :html-html5-fancy t :html-head-include-default-style nil :html-head-include-scripts nil :htmlized-source t :html-head-extra ,org-blog-head :html-preamble org-blog-preamble :html-postamble org-blog-postamble :auto-sitemap t :sitemap-filename "archive.org" :sitemap-title "Blog Posts" :sitemap-sort-files anti-chronologically :sitemap-format-entry org-blog-sitemap-format-entry :sitemap-function org-blog-sitemap-function ) ("blog-static" :base-directory "~/org/posts/" :base-extension "jpg\\|png\\|css\\|js\\|ico\\|gif\\|pdf\\|ogg" :recursive t :publishing-directory "~/org/blog/" :publishing-function org-publish-attachment ) ("blog-rss" :base-directory "~/org/posts/" ;; :base-extension "org" :html-link-home "http://reverland.org" :html-link-use-abs-url t :rss-extension "xml" :publishing-directory "~/org/blog/" :publishing-function (org-rss-publish-to-rss) :exclude ".*" :include ("archive.org") :section-numbers nil :with-toc nil :title "Reverlands Playground") ("blog" :components ("blog-content" "blog-static" "blog-rss")))) ) (provide 'org-blog)
and bind a command to some hotkeys
(use-package org-blog :bind (("C-c b" . publish-blog)) )
Easy New Blog Post with Capture Template
add this, and M-x org-capture RET to add new blog files.
(defun org-blog-create-posts-file () "Create an org file in ~/org/posts/." (interactive) (let ((name (read-string "Filename: "))) (expand-file-name (format "%s.org" name) "~/org/posts/"))) (setq org-capture-templates '( ("b" "New Post" plain (file org-blog-create-posts-file) (file "templates/blog-post.tpl"))))
Migrate disqus thread
use disqus url mapping disqus migration tools.
with open("reverlandblog-2019-11-11T03:07:50.790956-links.csv") as f: s = f.read() lines = [l for l in s.split("\r\n") if l != ""] for l in lines: l = l.split("?")[0] x = l.replace('http://reverland.org/', '').split("/") if x[0] == "about.html": print("%s,%s", (l, "https://reverland.org/index.html")) continue elif len(x) < 5: continue y = x[1] m = x[2] d = x[3] t = x[4] print("%s,%s" % (l, ("https://reverland.org/%s-%s-%s-%s.html" % (y, m, d, t)).lower()))
