So I have been learning a little bit of programming using the R language, and in a hurry I’ve become a big fan of the R Markdown file type for this. It is based on Markdown, which is the input syntax that our forum uses.
That last link gives a better description than I could (and provides an excellent table for learning how to enter characters to get the text you want here), but briefly, Markdown is a way to format documents so that the characters that you type writes and formats the text in your document, inserts links, and more – without having to take your hands off the keyboard.
It is really handy to learn it on the forum, and I have picked up a fair bit using the buttons and comparing the entry text and preview on the forum here. Besides the wikipedia link to Markdown that I shared above — and will relink here because it is still copied and I can use this as an example of how to encourage someone to click on a link with Markdown by making the text really long — the other thing I wanted to share is a REALLY handy guide specific to R markdown. Only the stuff on the front page will apply to the forum, but there are a couple of examples (such as table entry) that are not explained on wikipedia. Inserting the proper spaces, and line-breaks is really important, so testing stuff with the preview window on the forum here is great (make sure that your browser is full screen width so it shows up). So here is that link for the R Markdown guide (I waited a few lines to space it out from the block above).
Hopefully someone finds it helpful!
It has been a few years, but R is still one of my very favorite tools for turning massive dumps of data into usable conclusions.
If your application might need to scale big you probably should plan to GPU accelerate from the ground up… This stuff happens to be compatible with the computing arrays I’ve had to become familiar with at work:
If your application might need to scale big
Thanks for the resource Pierre, I’m primarily trying to learn things from the ground up in R, so while I plan to get to big data eventually, it is certainly not where I plan to start.
My first few projects have been using R markdown and the RMD Tufte style handout package to streamline writing. I’m happy to have cut MS Word out of my workflow (except for collaborating on documents), and I can go straight from writing using Markdown in my favourite text editor, compile RMD/citations in R Studio, and then have a fully formatted PDF or html code in a hurry. I’m working on sending html emails from R Studio right now, but I can currently only attach the .html file to the email, instead of having it appear in the body (probably not a big deal, because the Tufte package doesn’t inline the CSS anyhow, so many mail clients would strip it off anyhow. I will need to figure out code for that before I can send emails right from the compiler.
Next steps are working with the ggplot2 package to figure out some data visualizations with my thesis data set (I can and have ] done these in other programs already, but can get a better and more configurable product if I go to R, plus its a great learning experience). This dataset only has like 1000 birds and less than 200 data points for each, so its big enough for some solid visualizations/stats, but it shouldn’t push a computer too hard.
Thanks for sharing that resource!
Yeah, we learned the hard way that just because something worked fine when you had a few thousand rows in your table, that does not mean it will complete the calculation in a timely manner ten years down the road when the table contains 100Tb of data. I don’t even know how many rows it is, just that you can grind on it for days to find things out…