#33 ACG Website: move from HostMonster to GitHub, layout and settings

RedshiftDrift2023-06-08, edited 2023-10-07OpenA Cosmology Group

help wanted

I am trying to find the quickest way to change the appearance of the website so that there will be an index of the site in a separate column that will show the folder structure.

I started reading the manual Quickstart for GitHub Pages where they have the settings for the webpage in the file _config.yml. However in our repo the file is called _config.yaml.

Is there a reason for the different name?

Comments (39)

RedshiftDrift2023-06-08, edited 2023-06-08

Never mind, I found the answer!

ftessier2023-06-08, edited 2023-06-08

I think both extensions work (?) I don’t even know which one I prefer 😄, .yaml is certainly unequivocal.

👍 1

ftessier2023-06-08, edited 2023-06-08

I think the easiest is to find a remote Jekyll theme and apply it, for example: https://jekyllthemes.io/free

ftessier2023-06-08

This one looks like an interesting starting point: https://jekyllthemes.io/theme/just-the-docs

2👍 1

RedshiftDrift2023-06-08, edited 2023-06-08

Not all themes are supported in GitHub, this list https://pages.github.com/themes/ seems accurate.

ftessier2023-06-08, edited 2023-06-08

Ah, I see. I know very little about Jekyll, having only used the default Slate theme in the past. It seems possible to build the (static) Jekyll site offline, commit the files and serve them on github.io, but I don’t know enough about this to advise at this point.

I would argue it is most important to keep things automated upon commits, so that we can forget about the web page serving side of things (once a proper theme is enabled), and focus on just the content in the .md files.

👍 1

sahil5d2023-06-08, edited 2023-06-08

https://just-the-docs.com @marmetl this does work with GitHub Pages

👍 1🎉 1

RedshiftDrift2023-06-09

This looks indeed like a good theme for a-cosmology-group… abit like GitBook. Thanks @sahil5d I will explore this.

😄 1

RedshiftDrift2023-06-11, edited 2023-06-11

I read more documentation about just-the-docs, Jekyll, etc… and a few things need to be changed in the repo to get this to work. I will try to get it to work (beyond the bare-minimum template 😄 )

Expect many commits as I move files, build my first template and test the output.

ftessier2023-06-11, edited 2023-06-11

No worries. You can experiment on a side branch, but I guess if you want to see the changes you’ll want to commit to main? Unless you can serve from another branch. You know more than me. But we can always clean up the main branch afterwards if you want, I can certainly help with that.

RedshiftDrift2023-06-11, edited 2023-06-11

Yes I’ll test from the main branch directly to see the changes (to see if it works in the first place!) just-the-docs published from the docs directory, so I’ll be changing that too.

Expect a few compile errors…

RedshiftDrift2023-06-12

I disabled pages so I wouldn’t get error messages every time I commit. That didn’t work, it’s complaining that pages is disabled! (There must be another way to disable this, but this may be in Actions which I don’t understand yet.)

So I tried to enable pages again, but it says to contact the organization administrators. What am I, an owner, a maintainer but not an administrator?

Could you either give me permissions to enable pages, or if you enable this yourself, deploy from the main branch with the docs folder! 🙏 github-pages

RedshiftDrift2023-06-12

… and I created my own branch to upload all the files from the template and move some other files, I don’t need to see the changes until the entire template is in place. That will help making the a-cosmology-group main-branch less messy 😄

ftessier2023-06-12, edited 2023-06-12

🤔 Beats me. You are owner on the organization and admin on the repo! It says the same thing for me, but I cannot look into this right now and might not for the week (teaching EGSnrc course in Wisconsin!). Will take a jab at it as soon as possible. That is extremely annoying. I presume that something went wrong because we activated pages while the repo was a separate entity, then we amalgamated the repo and the org (Discussions, maybe settings, access?).

ftessier2023-06-12, edited 2023-06-12

Found it!

Under the org settings -> Member privileges:

This was checked off. This is an org level setting to control whether org members can publish websites from the repos they create. Should be good now!

RedshiftDrift2023-06-12, edited 2023-06-12

It works!

Thanks! Your course has top priority, don’t worry, ACG can wait!

The universe is older than 35 billiion years, one more day won’t make any difference!

😄 1

RedshiftDrift2023-06-12, edited 2023-06-12

Success! https://a-cosmology-group.github.io/acg/ Still a bit of tweaking to do (each page needs a YAML front matter) but otherwise it seems to work.

🚀 2

ftessier2023-06-12

Beautiful @marmetl!! I was wondering how to do that, I wanted something similar for the EGSnrc documentation. Thank you for showing the way. 🙏🏻

I can help convert some of the legacy .html to .md in a fairly automatic fashion (using pandoc). Let me know if some files ought to be converted in priority.

👍 1🎉 1

RedshiftDrift2023-06-13, edited 2023-06-13

Thanks @ftessier for the ~automatic conversion. There is no real priority, the cosmology.info url will be moved to the GitHub website when it is mostly ready.

RedshiftDrift2023-06-13, edited 2023-06-13

By the way @ftessier would it help you to have a .zip file of the entire website? I could post it on GoogleDrive…

ftessier2023-06-13, edited 2023-06-13

This is provided by github too! You can do that with a “Release”. See the EGSnrc releases for an example.

When you create a “Release”, the entire repository is attached as both a .zip and a .tar.gz, under the “assets” section. Anyone can download the compressed archive(s).

You can structure Releases as you wish. For a website, it stands to reason that you would one maintain a single “Release”, which would always be kept up to date.

Let me know if you want to experiment with that right away, or set this aside for a rainy day!

RedshiftDrift2023-06-13, edited 2023-06-13

Yes, I already used it 😉. I didn’t express myself clearly… I meant a .zip file of the entire cosmology.info website that I could post it on GoogleDrive…

ftessier2023-06-13, edited 2023-06-13

I still don’t understand… Ah, you mean a zip of the original website, right? Yes, it is a good idea to keep this around, especially over the transition, if anyone wants to check on a bit that might not have been converted properly. I would attach the .zip file to a Discussion here though, so it is all associated with the repo.

ftessier2023-06-13

Or, if it is not too large (strip .pdf and other binaries?), we can commit it inside the repo itself, for the record. But since the files therein won’t change, there is little value in versioning them, apart from access in the repo itself.

RedshiftDrift2023-06-13

No, I am still not getting my message through to you!

All the files for the website cosmology.info (the original, not the GitHub version) are on my computer. Copies of those files are on the cosmology.info server, so they are seen on the website cosmology.info. In the past weeks, you have been taking files from cosmology.info, converting them with pandoc and committing them to the a-cosmology-group repo.

I am offering to:

Generate a cosmology_info.zip file of the original website from my computer with everything in it (contains .html, .jpg, .pdf, .csv, .txt and .zip),
Post cosmology_info.zip on my Google drive and give you (only you) access to it for your private use.

This would allow you to:

Download the cosmology_info.zip file it to your computer (not to a Git repo, no versioning of that file is required),
Unzip cosmology_info.zip to have access to the folder structure of cosmology.info on your computer.

So far, nothing has been committed on GitHub yet 😄 ! Now, you could

Pick the cosmology_info files from your computer (instead of downloading them from the web) and convert the .html format to .md with pandoc,
Then upload the resulting .md files to the repo, as you have already done for some files.
Once you are done you can delete cosmology_info.zip.

I am offering this only if it makes easier for you to work this way instead of getting the files from the cosmology.info website.

The whole site is 103 MB big, most of the space is for files that won’t change, so there’s no need to commit a .zip file of the same website.

Only the .html files need to be converted to .md and committed to the a-cosmology-group/acg repo. I can upload the other formats (.jpg, .pdf, .csv, .txt and .zip) directly to the a-cosmology-group quickly since no conversion is necessary. (See a-cosmology-group/acg/newsletters/assets, it took 10 seconds to upload that!)

ftessier2023-06-13, edited 2023-06-13

Oh gosh @marmetl I am sorry I made you write this long explanation! The reason I seemed so clueless is because I already have all the files, and I assumed you knew that. So I automatically thought you meant “for other people”, not just me (even though you said “would it help you”). Apologies. I used the following terminal command to download the entire website:

wget -r http://cosmology.info/

👍 1

RedshiftDrift2023-06-14

No problem, I did not know that you could “download” the entire site 😮 with one command!

RedshiftDrift2023-06-14, edited 2023-06-14

I uploaded several 10s of MB of .pdf files and squash and merged the whole thing. Every file that’s not in .md format is placed in assets. README.md files are for information about the folder, while index.md files are for the website’s index.html.

I am not exactly following the folder structure of the original website. The folder models will take the webpages from individual members who want to create them. @ftessier you can merge Claudio README.md but rename it index.md if possible, and don’t merge the pdf file for now (if that’s possible). See my comment there.

The website https://a-cosmology-group.github.io/acg/ is posted correctly but some error messages are generated. I will look into that later.

ftessier2023-06-21, edited 2023-06-21

See my comment there.

@marmetl I am back from the course, but I have lost track a little of what is happening here. 😄 Can you point me to where there is?

RedshiftDrift2023-06-21, edited 2023-06-21

Welcome back! This is not relevant anymore, I merged Claudio’s branch (“there” was here in case you still want to read all that confusion 😄 )

ftessier2023-06-21, edited 2023-06-21

Alright, thanks for letting me off the hook. 🤓

If you want, don’t hesitate to prompt me ahead of a merge, so that I can “clean up” the pull request branch (typically squashing the many little “irrelevant” commits together). Up to you, makes it easier to browse the history, but has no impact on content per se.

If and when I do, I will detail the operations, so that everyone can learn how to do this!

RedshiftDrift2023-06-21, edited 2023-06-21

I use “squashing” when I merge many edits to many files. Can you clean this (e.g. this one) even more?

ftessier2023-06-21, edited 2023-06-21

Sorry, I had not seen that you already squashed these pull request branches, I see now from the commit history on main that the PRs were indeed squashed, all good. I cannot reduce it below one commit.

However, let me know if you want to cleanup the branch without necessarily squashing everything together. For example, you might want to get rid of some noise due to trial and error, but still retain 3 separate commits for 3 logically separate updates. It is possible to combine and reorder commits at will, mostly using the git rebase -i and git cherry-pick terminal commands. I do this every day, so I got the hang of it.

Otherwise, squash away as you have been doing so far! 👍🏻

👍 1

RedshiftDrift2023-06-21, edited 2023-06-21

Thanks, I’ll remember that you can do all that!

At this point I am still going through tests and none of the earlier commits are of any use. It’s not like developing code, these files are only webpages with typos, so you could squash the entire main acg branch and no useful information would be lost!

B.t.w. I did add some javascript code - I moved my cosmology calculator here 😄

👍 1

RedshiftDrift2023-06-24, edited 2023-06-24

For a week now I have been editing files on by computer and pushing with the terminal (non text-files can’t be renamed/moved using the web interface 😠 )

I use a limited set of commands:

$ git pull
$ git status

then I modify/edit several files, and then

$ git add .
$ git commit -m "Test Jekyll/just-the-docs/MathJax and edit website"
$ git push origin main

… and then I check the workflow run on the GitHub web interface and the updated website!

@ftessier I just realize that $ git push origin main doesn’t squash, so you may want to clean up the many commits done in the last week!

My question is, what would be the git command to push and squash? Is there something like $ git push --squash origin main I can use?

oposay2023-06-21, edited 2023-06-21

very informative

1👍 1

ftessier2023-06-25, edited 2023-06-25

@marmetl You are really getting the hang of this!

Indeed, working on the github interface (pull request, issues, etc.) is useful to interact with the community who submit ideas or bugs. But working with git on a local clone and pushing to the repo is much more convenient. There are software guis that are very nice to work locally, if the terminal is not your thing (VS Code with github integration, GitKraken, etc., there is a zoo of them).

For squashing in the terminal, the command you are looking for is git rebase --interactive [base], where [base] is the commit hash on which you want to “rewrite history”.

Git rebase is a slightly complex commands, as it is a two-step process. You are first presented with a list of commits from the rebase [base] to the current commit, in your text editor, in chronological order. For example, main is now 9da3c86, and if I git rebase -i 8000bb8, my editor opens with the following content (instructions included!):

pick 3ba123f Add gravity models navigation
pick 82c0dcc Test line breaks
pick a1b1d45 Fix equation
pick 0fe9c37 Add documents to
pick eb811c9 Fix web pages (#73)
pick 5104a6f Pull again? (#74) # empty
pick 9da3c86 Edit Non Expanding Cosmologies webpages

# Rebase 8000bb8..9da3c86 onto 8000bb8 (7 commands)
#
# Commands:
# p, pick <commit> = use commit
# r, reword <commit> = use commit, but edit the commit message
# e, edit <commit> = use commit, but stop for amending
# s, squash <commit> = use commit, but meld into previous commit
# f, fixup [-C | -c] <commit> = like "squash" but keep only the previous
#                    commit's log message, unless -C is used, in which case
#                    keep only this commit's message; -c is same as -C but
#                    opens the editor
# x, exec <command> = run command (the rest of the line) using shell
# b, break = stop here (continue rebase later with 'git rebase --continue')
# d, drop <commit> = remove commit
# l, label <label> = label current HEAD with a name
# t, reset <label> = reset HEAD to a label
# m, merge [-C <commit> | -c <commit>] <label> [# <oneline>]
#         create a merge commit using the original merge commit's
#         message (or the oneline, if no original merge commit was
#         specified); use -c <commit> to reword the commit message
# u, update-ref <ref> = track a placeholder for the <ref> to be updated
#                       to this position in the new commits. The <ref> is
#                       updated at the end of the rebase
#
# These lines can be re-ordered; they are executed from top to bottom.
#
# If you remove a line here THAT COMMIT WILL BE LOST.
#
# However, if you remove everything, the rebase will be aborted.
#

(3ba123f is the first commit after 8000bb8).

This is where you tell git how you want to rewrite the commit history: by changing the first word on the line for each commit (initially pick for all). In terms of squashing everything, you can change pick to squash (or simply s) for all commits except the first one (since you need a commit to squash “onto”.

Upon saving and quitting the editor, git will proceed with rewriting the history accordingly. When squashing, you will be presented with the opportunity to edit the squashed commit message, which by default is a concatenation of all the messages from the squashed commits.

There are many other options in rebase, like reordering commits (by reordering the lines), reword the commit message (useful for typos), edit commit to change the content of the commit (say, adding files or tweaking the commit content). Note however that it can lead to conflicts. If things get out of hand, you can always issue git rebase --abort to go back to the original branch (before the rebase was initiated). Squashing (without reordering) never leads to conflicts.

Finally, “rewriting history” is not recommended if you have already shared the commits publicly: collaborators might have started forks or modifications on the existing commits. Due to the “cryptographic blockchain” nature of the commit references (hashes of changes, metadata, and the parent commit hash), rebasing implies that all rebased commits will acquire a new commit reference. That is why it is a good practice to work on side-branches, and merge the resulting commit or branch into main, which itself is never rebased so that refs are permanent and always valid on main.

1👍 1

RedshiftDrift2023-06-25

Thanks @ftessier , it works and generates the commands automatically which are executed once I close the file! That’s quite a powerful tool.

$ git rebase --interactive
hint: Waiting for your editor to close the file...
unix2dos: converting file C:/Users/Louis.NEURONE10/Sync/Websites/a-cosmology-group/.git/rebase-merge/git-rebase-todo to DOS format...
dos2unix: converting file C:/Users/Louis.NEURONE10/Sync/Websites/a-cosmology-group/.git/rebase-merge/git-rebase-todo to Unix format...
Successfully rebased and updated refs/heads/main.

However, in practice, because I am testing Jekyll/just-the-docs/MathJax on the acg(main)-branch, I have only one commit to squash on my local repo. So I only had one command line and no squashing to do. I understand why “rewriting history” would not be a good idea.

This is way more than I signed up for! I only wanted to transfer the website and get this to work automatically after a push. I won’t be using this in the future, so unfortunately most of it will be forgotten in a few months… (I’ll have my notes.)

ftessier2023-06-27

This is way more than I signed up for! I only wanted to transfer the website and get this to work automatically after a push. I won’t be using this in the future, so unfortunately most of it will be forgotten in a few months… (I’ll have my notes.)

No doubt more than you signed up for! 😄 But throught it all you have taught me how to build a proper website on github, so thank you! I can remain on the sideline to help with git if needed.

The strategy I found to not forget git over time is to use it for all my projects. In roughly a decade of doing that, I have not yet found a situation where that has not been beneficial. It is a little cumbersome at first, but the work-add-commit-work-add-commit-… dance quickly becomes second nature.

👍 1