Migration to Org mode

Transform Markdown Files into Org files

Using bash script and pandoc to do this transform.

for f in `ls *.md`; do 
  echo "[*] process: $f"
  pandoc -f markdown -t org -o ${f%.md}.org ${f}; 
done

however, some error happened.

➜  blog for f in `ls *.md`; do 
for>   echo "[*] process: $f"
for>   pandoc -f markdown -t org -o ${f%.md}.org ${f}; 
for> done
[*] process: 2012-02-07-vim-latexxelatex.md
[*] process: 2012-02-23-vimlisp.md
[*] process: 2012-03-06-vbox.md
[*] process: 2012-03-11-vrdp.md
[*] process: 2012-03-21-lfs.md
[*] process: 2012-03-25-archlinux3264.md
[*] process: 2012-04-13-How-I-built-this-blog.md
[*] process: 2012-04-16-half-hackerhalf-artist.md
[*] process: 2012-04-24-.md
[*] process: 2012-04-26-a-text-game-engine-written-by-lisp.md
[*] process: 2012-04-28-visualize-tree-like-data-in-lisp.md
[*] process: 2012-05-01-grand-theft-wumpus.md
[*] process: 2012-05-03-the-orc-battle-game.md
[*] process: 2012-05-06-using-loop-to-evolve.md
[*] process: 2012-05-09-attack-of-the-robots.md
[*] process: 2012-05-10-streams-in-lisp.md
[*] process: 2012-05-12-web-server-written-in-lisp.md
[*] process: 2012-05-14-.md
[*] process: 2012-05-18-.md
[*] process: 2012-05-20-.md
[*] process: 2012-05-23-dice-of-doom.md
[*] process: 2012-05-26-.md
[*] process: 2012-05-28-archlinuxpdf.md
[*] process: 2012-05-31-tail-call-optimization.md
[*] process: 2012-06-02-memoization-in-functional-style.md
[*] process: 2012-06-05-scheme.md
[*] process: 2012-06-08-first-ai-in-lisp.md
[*] process: 2012-06-09-gimp.md
[*] process: 2012-06-12-.md
[*] process: 2012-06-13-macro-in-lisp.md
[*] process: 2012-06-16-dsl.md
[*] process: 2012-06-19-.md
[*] process: 2012-06-21-linux.md
[*] process: 2012-06-22-a-mathjax-test.md
[*] process: 2012-06-23-.md
[*] process: 2012-06-26-common-lisp.md
[*] process: 2012-06-28-google-glasses.md
[*] process: 2012-06-29-dice-of-doom-v2.md
[*] process: 2012-07-02-webdice-of-doom.md
[*] process: 2012-07-05-.md
[*] process: 2012-07-07-dice-of-doom-v4.md
[*] process: 2012-07-09-land-of-lisp.md
[*] process: 2012-07-11-.md
[*] process: 2012-07-13-.md
[*] process: 2012-08-04-.md
[*] process: 2012-08-11-.md
[*] process: 2012-08-15-galgame.md
[*] process: 2012-08-17-to-be-an-epicurean.md
[*] process: 2012-08-22-numpy.md
[*] process: 2012-08-24-scipy.md
[*] process: 2012-08-30-sympy-tutorial.md
[*] process: 2012-09-07-matplotlib-tutorial.md
[*] process: 2012-09-12-.md
[*] process: 2012-09-20-.md
[*] process: 2012-10-14-always-look-on-the-bright-side-of-life.md
[*] process: 2012-10-14-scikit-learn-machine-learning-in-python.md
[pandoc warning] Could not parse YAML header: did not find expected comment 
or line break "source" (line 41, column 3)                                 
[*] process: 2012-10-19-python.md
[*] process: 2012-10-22-scipy.md
[*] process: 2012-10-30-.md
[*] process: 2012-10-31-python.md
[*] process: 2012-11-12-numpyscipy.md
[*] process: 2012-11-19-flask.md
[*] process: 2012-12-02-python.md
[*] process: 2013-01-06-summary-and-prospect-in-2012.md
[*] process: 2013-01-13-moving-to-gentoo.md
[*] process: 2013-01-26-data-analysis-with-open-source-tools.md
[*] process: 2013-01-28-visualize-your-shell-history.md
[*] process: 2013-02-05-visualize-the-friendship-of-renren.md
[pandoc warning] Could not parse YAML header: did not find expected alphabet
ic or numeric character "source" (line 15, column 2)                       
[*] process: 2013-02-19-yet-another-photomosaic-generator.md
[*] process: 2013-02-25-generate-ascii-images-like-the-matrix.md
[*] process: 2013-03-02-python.md
[*] process: 2013-03-07-image-to-css.md
[*] process: 2013-03-11-quickstart-ebuild-guide.md
[*] process: 2013-03-13-advanced-python-constructs.md
[pandoc warning] Could not parse YAML header: did not find expected alphabet
ic or numeric character "source" (line 27, column 2)                       
[*] process: 2013-03-24-.md
[*] process: 2013-04-07-2-d-phase-unwrapping-path-following.md
[*] process: 2013-04-12-hello-bupt.md
[*] process: 2013-04-16-interfacing-with-c.md
[*] process: 2013-04-27-a-beautiful-night-in-jnu.md
[*] process: 2013-05-13-graduation-season.md
[*] process: 2013-05-16-some-books.md
[*] process: 2013-06-01-recent-summary.md
[*] process: 2013-06-09-hacking-the-art-of-exploitation.md
[*] process: 2013-07-02-goodbye-jnu.md
[*] process: 2013-07-22-build-up-your-own-pt-lab.md
[*] process: 2013-07-26-shellcode-black-magic.md
[*] process: 2013-08-03-heap-unlinking.md
[*] process: 2013-08-13-notes-on-anti-virus-evasion.md
[*] process: 2013-08-21-new-travel.md
[*] process: 2013-08-26-win32-shellcode.md
[*] process: 2013-08-31-so-boring.md
[*] process: 2013-09-06-.md
[*] process: 2013-09-09-.md
[*] process: 2013-09-13-.md
[*] process: 2013-09-15-.md
[*] process: 2013-09-30-.md
[*] process: 2013-10-11-.md
[*] process: 2013-10-15-wheres-the-way.md
[*] process: 2013-10-31-feelings-up-and-down.md
[*] process: 2013-11-09-what-a-fucking-life.md
[*] process: 2013-11-25-cloud-atlas.md
[*] process: 2013-12-21-.md
[*] process: 2013-12-24-writing-drivers-in-linuxa-brief-tutorial.md
[*] process: 2013-12-31-.md
[*] process: 2014-01-11-thanks.md
[*] process: 2014-01-12-python.md
[*] process: 2014-01-29-python.md
[*] process: 2014-02-18-.md
[*] process: 2014-02-26-spoofing.md
[*] process: 2014-03-04-play-arp.md
[*] process: 2014-03-07-play-dhcp.md
[*] process: 2014-03-15-play-icmp.md
[*] process: 2014-04-03-gns3-on-gentoo-linux.md
[*] process: 2014-04-18-.md
[*] process: 2014-05-04-360-hack-game.md
[*] process: 2014-05-04-.md
[*] process: 2014-05-10-gentoolaunchpad-msp430.md
[*] process: 2014-05-18-a-glance-at-d3js.md
[*] process: 2014-05-31-mysql-learning-notes.md
[*] process: 2014-06-02-.md
[*] process: 2014-06-13-.md
[*] process: 2014-07-26-upgrade-gentoo-linux.md
[*] process: 2014-08-07-radare2-shellcode.md
[*] process: 2014-08-08-better-and-beautiful-console.md
[*] process: 2014-08-08-defeating-ioli-with-radare2.md
[*] process: 2014-08-14-fbi-tor-malware-analysis.md
[*] process: 2014-08-26-.md
[*] process: 2014-09-08-escape-plan.md
[*] process: 2014-09-21-quora.md
[*] process: 2014-10-09-lrc.md
[*] process: 2014-10-09-.md
[*] process: 2014-10-11-gcjust-for-fun.md
[*] process: 2014-10-13-on-disk-encryption.md
[*] process: 2014-10-15-dwarf-fortress.md
[*] process: 2014-10-17-acl-on-linux.md
[*] process: 2014-11-12-some-articles.md
[*] process: 2014-11-15-.md
[*] process: 2014-11-17-linux-audit.md
[*] process: 2014-11-18-file-permissions-and-masks.md
[*] process: 2014-11-20-account-and-access-control.md
[*] process: 2014-12-24-.md
[*] process: 2015-03-02-2015s-first-post.md
[*] process: 2015-03-09-.md
[*] process: 2015-03-11-shellcodeenvironment.md
[*] process: 2015-03-16-shellcodeloaders.md
[*] process: 2015-03-16-shellcode.md
[*] process: 2015-03-17-shellcodedynamic.md
[*] process: 2015-03-18-load-time-relocation-of-shared-libraries.md
[*] process: 2015-03-19-position-independent-code-pic-in-shared-libraries.md
[*] process: 2015-03-30-boot-sequence.md
[*] process: 2015-04-01-python-spider-quick-and-dirty-way.md
[*] process: 2015-04-02-baidu-music.md
[*] process: 2015-04-03-ydcx.md
[*] process: 2015-04-04-chaoxing.md
[*] process: 2015-04-05-yingxin.md
[*] process: 2015-04-07-python-spider-camera-find.md
[*] process: 2015-05-02-javascript.md
[*] process: 2015-05-10-.md
[*] process: 2015-05-17-17.md
[*] process: 2015-05-24-hacking-security.md
[*] process: 2015-06-01-security-in-browsers.md
[*] process: 2015-06-12-.md
[*] process: 2015-06-27-.md
[*] process: 2015-07-03-.md
[*] process: 2015-08-07-maybe-the-last-post-here.md
[*] process: 2015-11-12-i-am-back.md
[*] process: 2015-11-17-greasemonkey-userscript.md
[*] process: 2015-11-18-notes-on-migrating-from-jeykll-to-hexo.md
[*] process: 2015-11-21-the-es6-features-i-use-these-days.md
[*] process: 2015-11-23-linux-tex.md
[*] process: 2015-11-24-web-based-input-method.md
[*] process: 2015-11-29-using-buptgraduatethesis-on-ubuntu.md
[*] process: 2015-12-06-freedom.md
[*] process: 2015-12-14-wangyin.md
[*] process: 2015-12-20-stream-in-nodejs.md
[*] process: 2016-01-02-2015-summary-and-prospect.md
[*] process: 2016-01-15-webchat-user-bot.md
[*] process: 2016-02-09-play.md
[*] process: 2016-02-26-evolving-game-once-more.md
[*] process: 2016-02-28-say-something.md
[*] process: 2016-03-07-neural-network-101.md
[*] process: 2016-03-08-harvest-mm-pics.md
[*] process: 2016-03-09-black-hat-javascript.md
[*] process: 2016-03-22-yet-another-new-journey.md
[*] process: 2016-04-20-after-10-o-clock.md
[*] process: 2016-05-02-the-quieter-you-become-the-more-you-are-able-to-hear
.md                                                                        
[*] process: 2016-05-25-for-freedom.md
[*] process: 2016-06-11-writing-something-useful.md
[*] process: 2016-06-26-goodbye.md
[*] process: 2016-07-24-exploring-webpack.md
[*] process: 2016-09-13-half-year-full-time-front-end-developer.md
[*] process: 2016-11-02-interview.md
[*] process: 2017-01-08-spring-and-chaos.md
[*] process: 2017-05-26-memorize-bbs-byr-cn.md
[*] process: 2017-06-14-ubuntu-gnome-guide-for-mbp-users.md
[*] process: 2017-06-17-develop-a-mind-which-does-not-abide-in-anything.md
[*] process: 2018-01-06-2018.md
[*] process: 2018-11-11-dwarf-fortress-story-calendar.md
[*] process: 2019-06-24-i-am-back-again.md
[*] process: 2019-08-11-eink-experience-in-two-weeks.md
➜  blog 

The error occured.

process 2012-10-14-scikit-learn-machine-learning-in-python.md
[pandoc warning] Could not parse YAML header: did not find expected comment or line break "source" (line 41, column 3)

obselete old toc implement when i use jekyll, which I write as

— *目录 {:toc} —

Insert Back FrontMatters

I use a python script to migrate jekyll yml frontmatter to orgmode options

FROM = "/home/reverland/blog-next/src/posts/"
TO = "/home/reverland/tmpblog/"
for f in os.listdir(FROM):
    if f.endswith(".md"):
        fm = frontmatter.load(os.path.join(FROM, f))
        title = fm['title']
        date = f[:10]
        summary = fm['excerpt']
        tags = " ".join(fm['tags'])
        front = "#+TITLE: %s\n#+DATE: <%s>\n#+SUBTITLE: %s\n#+TAGS: %s\n\n" % (title, date, summary, tags)
        with open(os.path.join(TO, os.path.splitext(f)[0] + '.org')) as original:
            data = original.read()
        with open(os.path.join(TO, os.path.splitext(f)[0] + '.org'), 'w') as modified:
            modified.write(front + data)

Blog with Org mode

Org mode has a versatile export and publishing function.

I tweak a little from @narendraj9's org-blog.el.

(require 'org)
(require 'ox-publish)
(require 'ox-html)
(require 'org-element)
(require 'ox-rss)

(defvar org-blog-date-format "%h %d, %Y"
  "Format for displaying publish dates.")

(defun org-blog-prepare (project-plist)
  "With help from `https://github.com/howardabrams/dot-files'.
  Touch `index.org' to rebuilt it.
  Argument `PROJECT-PLIST' contains information about the current project."
  (let* ((base-directory (plist-get project-plist :base-directory))
         (buffer (find-file-noselect (expand-file-name "index.org" base-directory) t)))
    (with-current-buffer buffer
      (set-buffer-modified-p t)
      (save-buffer 0))
    (kill-buffer buffer)))

(defvar org-blog-head
  "<link rel=\"stylesheet\" href=\"https://stackpath.bootstrapcdn.com/bootstrap/4.1.3/css/bootstrap.min.css\" integrity=\"sha384-MCw98/SFnGE8fJT3GXwEOngsV7Zt27NXFoaoApmYm81iuXoPkFOJwJ8ERdknLPMO\" crossorigin=\"anonymous\">
  <link rel=\"stylesheet\" type=\"text/css\" href=\"/static/blog.css\"/>
  <link rel=\"stylesheet\" type=\"text/css\" href=\"https://fonts.googleapis.com/css?family=Amaranth|Handlee|Libre+Baskerville|Bree+Serif|Ubuntu+Mono|Pacifico&subset=latin,greek\"/>
  <link rel=\"shortcut icon\" type=\"image/x-icon\" href=\"/static/favicon.ico\">")

(defun org-blog-preamble (plist)
  "Pre-amble for whole blog."
  ;; Hack! Put date for the post as the subtitle
  (when (s-starts-with-p "20" (plist-get plist :input-file))
    (plist-put plist
               :subtitle (format "Published on %s"
                                 (org-export-get-date plist
                                                      org-blog-date-format))))

  ;; Return a simple banner with navigation links
  "<div class=\"banner\">
    <a href=\"/\"> Reverland的行知阁 </a>
  </div>
  <ul class=\"banner-links\">
    <li><a href=\"/archive.html\"> Posts </a> </li>
    <li><a href=\"/archive.xml\"> RSS </a> </li>
    <li><a href=\"/index.html\"> About Me </a> </li>
    <li><a href=\"/links.html\"> Links </a> </li>
  </ul>
  <hr>")

(defun org-blog-postamble (plist)
  "Post-amble for whole blog."
  (concat
  "<footer class=\"footer\">
    <a href=\"https://reverland.org\">
          <p> Built with
            <svg id=\"i-heart\" viewBox=\"0 0 32 32\">
          <path d=\"M4 16 C1 12 2 6 7 4 12 2 15 6 16 8 17 6 21 2 26 4 31 6 31 12 28 16 25 20 16 28 16 28 16 28 7 20 4 16 Z\"/>
            </svg> in
        <img id=\"i-emacs\" src=\"https://www.gnu.org/software/emacs/images/emacs.png\"/>
        <span id=\"view-source-link\"> copyleft@2012-2019 </span>
          </p>
    </a>
  </footer>
  <script type=\"text/javascript\" src=\"/static/blog.js\"> </script>

<!-- Global site tag (gtag.js) - Google Analytics -->
<script async src=\"https://www.googletagmanager.com/gtag/js?id=UA-38681562-1\"></script>
<script>
  window.dataLayer = window.dataLayer || [];
  function gtag(){dataLayer.push(arguments);}
  gtag('js', new Date());

  gtag('config', 'UA-38681562-1');
</script>
"

   ;; Add Disqus if it's a post
   (when (s-contains-p "posts/" (plist-get plist :input-file))
     "<!-- Disqua JS --> 
<div id=\"disqus_thread\"></div>
<div id=\"disqus_thread\"></div>
  <script type=\"text/javascript\">
   var disqus_shortname = 'reverlandblog';
   (function() {
     var dsq = document.createElement('script'); dsq.type = 'text/javascript'; dsq.async = true;
     dsq.src = '//' + disqus_shortname + '.disqus.com/embed.js';
     (document.getElementsByTagName('head')[0] || document.getElementsByTagName('body')[0]).appendChild(dsq);
   })();
  </script>
  <noscript>Please enable JavaScript to view the <a href=\"http://disqus.com/?ref_noscript\">comments powered by Disqus.</a></noscript>
")))

(defun org-blog-sitemap-format-entry (entry _style project)
  "Return string for each ENTRY in PROJECT."
  (when (s-starts-with-p "20" entry)
    (format "@@html:<span class=\"archive-item\"><span class=\"archive-date\">@@ %s @@html:</span>@@ [[file:%s][%s]] @@html:</span>@@"
            (format-time-string "%h %d, %Y"
                                (org-publish-find-date entry project))
            entry
            (org-publish-find-title entry project))))

(defun org-blog-sitemap-function (title list)
  "Return sitemap using TITLE and LIST returned by `org-blog-sitemap-format-entry'."
  (concat "#+TITLE: " title "\n\n"
          "\n#+begin_archive\n"
          (mapconcat (lambda (li)
                       (format "@@html:<li>@@ %s @@html:</li>@@" (car li)))
                     (seq-filter #'car (cdr list))
                     "\n")
          "\n#+end_archive\n"))

(defun org-blog-publish-to-html (plist filename pub-dir)
  "Same as `org-html-publish-to-html' but modifies html before finishing."
  (let ((file-path (org-html-publish-to-html plist filename pub-dir)))
    (with-current-buffer (find-file-noselect file-path)
      (goto-char (point-min))
      (search-forward "<body>")
      (insert (concat "\n<div class=\"content-wrapper container\">\n "
                      "  <div class=\"row\"> <div class=\"col\"> </div> "
                      "  <div class=\"col-sm-6 col-md-8\"> "))
      (goto-char (point-max))
      (search-backward "</body>")
      (insert "\n</div>\n<div class=\"col\"></div></div>\n</div>\n")
      (save-buffer 0)
      (kill-buffer))
    file-path))

(use-package ox-publish
  :requires (org ox-html ox-rss)
  :commands publish-blog
  :config
  (defun publish-blog ()
    (interactive)
    (org-publish-project "blog" t))

  (setq org-confirm-babel-evaluate nil)

  (setq org-publish-project-alist
        `(("blog-content"
           :base-directory "~/org/posts/"
           :exclude ".*~"
           :base-extension "org"

           :publishing-directory "~/org/blog/"

           :recursive t
           :preparation-function org-blog-prepare
           :publishing-function org-blog-publish-to-html

           :with-toc nil
           :with-title org-export-with-title
           :with-date org-export-with-date
           :section-numbers nil
           :html-doctype "html5"
           :html-html5-fancy t
           :html-head-include-default-style nil
           :html-head-include-scripts nil
           :htmlized-source t
           :html-head-extra ,org-blog-head
           :html-preamble org-blog-preamble
           :html-postamble org-blog-postamble

           :auto-sitemap t
           :sitemap-filename "archive.org"
           :sitemap-title "Blog Posts"

           :sitemap-sort-files anti-chronologically
           :sitemap-format-entry org-blog-sitemap-format-entry
           :sitemap-function org-blog-sitemap-function
           )
          ("blog-static"
           :base-directory "~/org/posts/"
           :base-extension "jpg\\|png\\|css\\|js\\|ico\\|gif\\|pdf\\|ogg"
           :recursive t
           :publishing-directory "~/org/blog/"
           :publishing-function org-publish-attachment
           )
          ("blog-rss"
           :base-directory "~/org/posts/"
           ;; :base-extension "org"

           :html-link-home "http://reverland.org"
           :html-link-use-abs-url t

           :rss-extension "xml"

           :publishing-directory "~/org/blog/"
           :publishing-function (org-rss-publish-to-rss)

           :exclude ".*"
           :include ("archive.org")
           :section-numbers nil
           :with-toc nil
           :title "Reverlands Playground")
          ("blog"
           :components
           ("blog-content" "blog-static" "blog-rss"))))
  )

(provide 'org-blog)

and bind a command to some hotkeys

(use-package org-blog
  :bind (("C-c b" . publish-blog))
  )

Easy New Blog Post with Capture Template

add this, and M-x org-capture RET to add new blog files.

(defun org-blog-create-posts-file ()
  "Create an org file in ~/org/posts/."
  (interactive)
  (let ((name (read-string "Filename: ")))
    (expand-file-name (format "%s.org"
                                name) "~/org/posts/")))
(setq org-capture-templates
      '(
        ("b" "New Post" plain (file org-blog-create-posts-file)
         (file "templates/blog-post.tpl"))))

Migrate disqus thread

use disqus url mapping disqus migration tools.

with open("reverlandblog-2019-11-11T03:07:50.790956-links.csv") as f:
    s = f.read()
lines = [l for l in s.split("\r\n") if l != ""]
for l in lines:
    l = l.split("?")[0]
    x = l.replace('http://reverland.org/', '').split("/")
    if x[0] == "about.html":
        print("%s,%s", (l, "https://reverland.org/index.html"))
        continue
    elif len(x) < 5:
        continue
    y = x[1]
    m = x[2]
    d = x[3]
    t = x[4]
    print("%s,%s" % (l, ("https://reverland.org/%s-%s-%s-%s.html" % (y, m, d, t)).lower()))