Love it or hate it, email is an inevitable part of modern communication. According to a 2017 study:There are 3.7 billion email accounts worldwide.We send 269 billion emails every day.On average, office workers receive 121 business emails and send 40 every work day.Two thirds of emails are read on mobile devices.But this was not always the case!Primitive electronic messaging has been around since the Computer Stone Age (aka the 1960s). The earliest versions of email allowed users to log in to an account and send messages to other users on the same computer. Imagine an electronic version of an old-school message board. While this was surely amazing technology at the time, it was obviously extremely limiting to have all messages tied to a single computer.In 1969, the Advanced Research Projects Agency Network (ARPANET) was developed to connect networks within research institutions, among many other tasks. Many consider ARPANET the precursor to the modern internet.Two years later, a programmer named Ray Tomlinson realized he could incorporate the code for single-computer messaging systems into the technology that allowed computers to communicate with each other through ARPANET. With this epiphany, users could now send emails to remote mailboxes on other computers.Ray Tomlinson sent the first email between these two computers in 1971.Sadly, as seems to happen with obsolete tech, both the content and the code behind the first email has been lost. Tomlinson claimed to not remember the first message he sent, but guessed it was something along the lines of “QWERTYUIOP.”After successfully sending a series of test messages between these two machines, Tomlinson emailed his colleagues to explain his new creation. In a super-meta twist, the first network email was used to announce the invention of email.https://medium.com/media/135d052875319068c3f4758f400cae4f/hrefFun Facts about @One hurdle Tomlinson had to overcome was how to connect a user’s mailbox name with network name. He chose @, which up to that point had been used in English to mean ‘at the rate of’ (e.g., 5 apples @ $1. Bargain!). In Tomlinson’s words, “I used the ‘at’ sign to indicate that the user was ‘at’ some other host rather than being local.”“I was mostly looking for a symbol that wasn’t used much,” Tomlinson said in an interview with Smithsonian. “And there weren’t a lot of options — an exclamation point or a comma. I could have used an equal sign, but that wouldn’t have made much sense… [I chose @ ] probably saving it from going the way of the ‘cent’ sign on computer keyboards.”Thanks to Tomlinson, this once-obscure symbol is now one of the most frequently used characters online.While English speakers refer to @ as the “at sign,” other cultures have developed nicknames for the now ubiquitous symbol:In Germany, Poland, and South America, it’s referred to as “monkey’s tail.”In China, it’s “little mouse.”French and Italians call it “snail.”In Finland, it’s called “miukumauku,” which translates to “sign of the meow.” They believe it looks like a curled up cat. @@@@WWWWW!‘An act of design of extraordinary elegance’In 2010, MoMA’s Department of Architecture and Design acquired the @ symbol for its collection, with the following explanation:The appropriation and reuse of a pre-existing, even ancient symbol — a symbol already available on the keyboard yet vastly underutilized, a ligature meant to resolve a functional issue (excessively long and convoluted programming language) brought on by a revolutionary technological innovation (the Internet) — is by all means an act of design of extraordinary elegance and economy. Without any need to redesign keyboards or discard old ones, Tomlinson gave the @ symbol a completely new function that is nonetheless in keeping with its origins, with its penchant for building relationships between entities and establishing links based on objective and measurable rules — a characteristic echoed by the function @ now embodies in computer programming language. Tomlinson then sent an email about the @ sign and how it should be used in the future. He therefore consciously, and from the very start, established new rules and a new meaning for this symbol.This simple device, a symbol we take for granted every day, was actually a deliberate, functional innovation, embodying an elegance to which all programmers should aspire.Sources:MoMA | @ at MoMAWonder How Many Emails Are Sent Worldwide Every Day?History of Computers and Computing, Internet, Birth, The First E-mail Message of Ray TomlinsonThe Accidental History of the @ SymbolThe First Email
In my previous blog post I attempted to create a visual representation of object oriented programming principles. As I dive deeper into CS fundamentals, I have found visual representations of sorting algorithms helpful in understanding the steps necessary to transform an unsorted input into a sorted output. It made me wonder if a sonic representation of a sorting algorithm at work would be instructive.I set out to model three sorting algorithms with sound: bubble sort, selection sort and insertion sort. In each case, I created an array of frequencies corresponding to a three-octave scale running up in C major. I then randomized the order of this array. After each iteration of the sorting algorithm, all of the notes are played in the order representing the current state of the array.Below are brief explanations of the sorting algorithms I modeled with sound, along with links to view and listen to these representations. I chose three simple and relatively similar algorithms to compare. If you listen through these a couple of times, you can start to pick out the unique audio profile of each algorithm. Ultimately, this exercise was more interesting for the sounds it resulted in than it was effective as a tool for understanding sorting algorithms.Bubble SortAudioization can be found here.Bubble sort is a simple but inefficient sorting algorithm, best used in cases where the input should already be sort and the task is to check that this is in fact the case. The algorithm compares each adjacent item in the input, and if they are out of order, it swaps their positions. Once it reaches the end of the input, it starts over. It repeats these steps until no swaps need to be made, indicating that the input is now ordered. Larger elements in the input “bubble” up to the top and as a result, the end of the input gets sorted before the beginning. See if you can hear the higher frequency notes getting ordered first.Best case scenario, when the input is already sorted, bubble sort has a complexity of O(n). Worst case scenario, when the input is in reverse order, bubble sort has a time complexity of O(n²). Unfortunately, O(n²) is also the average time complexity. Bubble sort should really only be used is situations where one needs to confirm the sortedness of an input but not to do the sorting itself.You will notice that the bubble sort happens faster than the insertion or selection sort. This is not a result of the efficiency of the bubble sort. In fact, it is the least efficient of the three. Instead, this happens because bubble sort makes the most swaps per cycle through the entire input and the sounds are played only after each cycle. Bubble sort does more work per cycle, but performs fewer cycles.Insertion SortAudioization can be found here.Like bubble sort, insertion sort is both simple and inefficient. It is best used only in situations where the input size is extremely small. Starting from the beginning of the input, insertion sort removes and item and compares it to each item of lower index until it finds the place where the item is smaller than input item to the right, and larger than the input item to the left. It repeats this process until it gets to the end of the input at which point the input will be sorted. Unlike bubble sort, insertion sort grows in sortedness from the bottom of the input. See if you can hear this in the audioization.Like bubble sort, the best case scenario time complexity for insertion sort is O(n). Also, like bubble sort, the worst case and average time complexities are quadratic. That said, with very small input sizes insertion sort is very fast. What “very small” means exactly depends on the machine the algorithm is running on, but it is typically an input with a size of 10 items or less. Implementations of quicksort will often include a check on the input size that will simply run an insertion sort instead of the quicksort if the size is small enough.Selection SortAudioization can be found here.Selection sort is another simple and inefficient sorting algorithm. In selection sort, the input is divided into a sorted region and an unsorted region. The unsorted region is searched for the smallest item, which is then placed at the end of the sorted region, growing the sorted region and shrinking the unsorted region by one item. In bubble sort and insertion sort you could say that over time, items are sorted relative to one another until the entire input is fully sorted. Selection sort is different in that there is a region of absolute (rather than relative) sortedness and a region that is fully unsorted.Selection sort has a best case, worst case and average time complexity of O(n²). Despite this, selection sort is usually faster than bubble sort, which has a best case complexity of O(n). This is because bubble sort has to compare every element to the next through every iteration, whereas selection sort has to compare fewer and fewer elements as the sorting progresses. Selection sort and insertion sort perform pretty similarly in terms of time complexity. Selection sort always performs O(n) swaps and therefore will typically perform fewer swaps than insertion sort. That said, insertion sort only scans elements until it finds the correct placement for the item that is being sorted, whereas selection sort has to check each item in the unordered sublist in order to confirm that it has found the next smallest item from the input.
At least Medium seems to think so.(I tried to write a clever title with a “million” spaces but Medium doesn’t allow consecutive spaces.)Why am I talking about whitespace?As innocuous as it seems, whitespace was one of the things that put me off programming in high school. Computer science was a mandatory at my high school were I was forced to learn Python. If you know anything about Python, it is one of the more infamous ‘whitespace significant’ programming languages. print "Hello World!"print "coffee"print "pizza"In Ruby, this would be not a problem. It would easily print out the text. However in Python:Traceback (most recent call last): File "python", line 1 print "Hello World!" ^IndentationError: unexpected indentIndentation Error? Really?As programmers, we have enough errors to deal with, whether it’s a slight misspelling of a variable or method (intialize? initilize? initalize?), an insignificant indentation is another error we don’t need to deal with. And it seems like I’m not the only one who feels this way.source: xkcd.comA fight as old as …typewriters?“The debate that refuses to die…” — James FeliciThe talk about spaces goes back to when to typists would use two spaces after a sentence to imitate classic typesetting. While the debate about spacing after the period of a sentence has been sidelined, it continues in digital world of computer programming text.source: Silicon Valley (HBO)Tabs vs Spacessource: xkcd.comVisiting any site discussing the merits of each programmers preferred language, the topic of whitespace is bound to come up. Some programmers dislike Python and other “off-side rule” languages because of whitespace significance. “Off-side rule” languages refer to languages such as Python, CoffeeScript, Cobra where code blocks are indicated by their indentations by the respective language. Many, like Cueball (above), are hesitant about some programming languages because of the forced indentation, the lack of a visual “end” of code, and it can definitely be just another nuisance rule in programming but whitespace is important no matter which language you program in.an actual article…Link: https://stackoverflow.blog/2017/06/15/developers-use-spaces-make-money-use-tabs/Surveying programmers from a variety of programming languages shows that spacing is actually slightly more preferred than using tabs.“The model estimated that using spaces instead of tabs is associated with an 8.6% higher salary. Put another way, using spaces instead of tabs is associated with as high a salary difference as an extra 2.4 years of experience.”***However, there is a computing advantage of using tabs over spaces. Tabs use less key presses therefore taking up less overall space in file size. But the space saving is minute, a few bytes at most. So using either tabs or spaces each have their own advantages but the real question is:Does it[either advantage] matter?***Remember: correlation is no causation.Whitespace makes code cleanOne of the first things you learn in programming is “indent you code”. Indenting your code, to the code interpreter, may just be stylistic, but a “stylized” code equates to:code that is easy to understand for othersIn the real world, you won’t be the only one to look at your code. Everyone has a different style when it comes to writing code, but for others reading your code, they should have no problem almost immediately pointing out where the code blocks are and where they start and end.code that is easier to readReadability means easy to locate and debug your code. The interpreter can only do so much as telling you the location where the code is bugging, but finding the precise location where your code is erring to fix is up to you. If you can’t read your code easily, you can’t debug easily.code that is easier to change and maintainIndentation helps delineate where your code block is. If you had to move or modify your code, there is an ease of doing so because of thanks to whitespace framing exactly where your code is.Your codecode that saves you time “Coders are lazy”, so I have been told. You should be able to just take a glance at code and know immediately where one code block starts and ends. You can ignore everything else and just easily focus on what you need to work on.Languages that are not whitespace sensitive, actually are?The Code Does Not Functiondef hello puts "Hello World!" endThis simple method Ruby seems fine enough but if you were to actually run this code, you would run into a syntax error. In Ruby, a new line is equal to a semiclone, which indicates the end of a line of code. Semicolons are optional in Ruby but who would want a code that looks like this:def hello; puts "Hello World!"; end;Who has time for semicolons?So while many claim Ruby to be whitespace insensitive, it actually seems indentations and newlines are important to have a functioning code.Incorrect Returns10+ 2==> 10Ruby says it doesn’t care about whitespace, so why am I getting incorrect returns? As mentioned, Ruby takes newlines are the end of the code line, so the interpreter sees the above line of code as:10;+2==> 12hence the incorrect result.A newline may not look like whitespace, but in regular expression specification, a newline is considered to be in the class of line terminators, part of what regex considers whitespace.Whitespace Programming LanguageWhitespace was created by Edwin Brady and Chris Morris at the University of Durham. It was released on April 1st, 2003 (April Fool’s Day). Compared to almost every other programming language, where whitespace is ignored by the interpreter, Whitespace ONLY recognizes whitespace inputs (spaces, tabs, line breaks). Every other character is ignored.This block of code returns “Hello, World!” Trust me.Conclustion: Empty Space is ImportantRemember:A hydrogen atom is 99.9999999999996% empty.The universe is mainly composed of hydrogen.The ever expanding universe is mostly empty space.In the beginning, the very very beginning of the Big Bang, the universe was creating from nothing.And from that nothing, we have everything.And similarly from nothing, we’ve learned to code, creating the foundation to be able to create programs and applications of all sorts that we were only able to imagine doing, while also incorporating that emptiness, whitespace.
Natural Language Processing: The Very BasicsNatural Language Processing (NLP) is the way computers attempt to understand human speech, either written or verbal. Because natural language isn’t literal, computers need a more tools to understand it. Beneath the text itself is subtext: the context of the communication, relationship of the reader and writer, unspoken connotations of words, cultural and/or geographic variations in meaning, humor and sarcasm, exaggeration, slang, etc.Part-of-Speech taggingOne of the most basic ways computers try to understand natural language is through Part-of-Speech tagger. A POS tagger, like EngTagger, uses context as well as a word itself to identify its part of speech. For example, let’s try the engtagger gem with the first phrase of the first sentence of Alice in Wonderland:“Alice was beginning to get very tired of sitting by her sister on the bank”Here’s what it looks like tagged:Alice/NNP was/VBD beginning/VBG to/TO get/VB very/RB tired/JJ of/IN sitting/VBG by/IN her/PRPS sister/NN on/IN the/DET bank/NNThis looks messy, but it’s pretty straight forward. Alice is recognized as a proper noun (NNP), was is a past tense verb (VBD), beginning is a gerund (VBG), and so on. Because the same word can be different parts of speech in different contexts, taggers first look at sentences as a whole, and tag words based on the words around them.KeywordsAnother way of breaking down text is through keyword analysis. If you tried to determine what a piece of writing was about based simply on the frequency of words, you’d end up with a lot of “the”s and “it”s. Keyword analyzers disregard articles and, depending on their sophistication, other words irrelevant to the meaning. Using the gem ‘highscore,’ I tried to determine the the top 10 keywords in the first part of Pride and Prejudice:Bingley - 336.0her - 252.0Miss - 231.0Elizabeth - 225.0Darcy - 216.0Bennet - 213.0was - 205.0Mrs - 156.0Jane - 147.0his - 139.0Ideally, we probably wouldn’t be interested in words like “her” and “was,” and we’d want it to know that Jane is the same person as Miss Bennet, but it does give us more information about the text.There are several other building blocks that NPLs use to classify and understand text, like lemmatization, syntactic parsing, and sentiment analysis. These all come together to form more sophisticated programs for understanding language.ImplementationI wanted to give this a try, so I used a simple naive Bayes classifier that used some of these methods behind the scenes to learn about text. First, I used Algorithmia’s html parser to get the first chapters of a few books. Then, I used these chapters to train my classifier on the difference between the author’s styles.author_cls.train(:Fitz, gatsby_chap_1)author_cls.train(:Twain, tom_sawyer_chap_1)author_cls.train(:Austen, p_and_p_part_1)Pretty simple. I tested it by asking it to classify quotes from outside the training text.# Sense and Sensibilityauthor_cls.classify("Know your own happiness.") #=> Austen# Gatsbyauthor_cls.classify("So we beat on, boats against the current, borne back ceaselessly into the past.") #=> Fitz# Huck Finnputs author_cls.classify("You can't pray a lie -- I found that out.") #=> FitzTwo out of three — for whatever reason, it didn’t want to classify anything as Mark Twain. Still, with a few helper gems, this took only a few minutes to put together.ApplicationsWho I Write Like is a more sophisticated (and open source!) program that works somewhat like the above. You can insert your own writing and see which of 50 authors most closely matches your style.Hemingway Editor helps you edit your writing by highlighting convoluted sentences, complicated words/phrases, adverbs, and passive voice.Harry Potter and the Portrait of what Looked Like a Large Pile of Ash is a chapter written in the style of Harry Potter using a predictive text algorithm.
“Machine learning” is all the buzz these days…and with reason. It has certainly infiltrated my daily life: when I see “People You May Know” on Facebook, when I ask Siri the weather, when I get a traffic prediction from Google maps, or when I am shown another ad for Allbirds on Instagram (and I finally decide to buy them). These are all awesome, helpful applications of machine learning — and they certainly make my life easier. However, I began to wonder what role machine learning plays in a larger, more systemic scope. As an engineer and former medical device rep, I was most interested in how machine learning is impacting medicine, as well as the healthcare industry.But before we dive into healthcare, since my initial understanding of machine learning was shaky at best, I’ll start with a high level overview:So, what exactly is machine learning? There are a lot of definitions out there, but I thought TechEmergence summarized it best:“Machine Learning is the science of getting computers to learn and act like humans do, and improve their learning over time in autonomous fashion, by feeding them data and information in the form of observations and real-world interactions.”Instead of explicitly programming computers how to do things, we train them to learn themselves. It is an incredibly general definition that can be applied to almost anything, healthcare included.Supervised vs. Unsupervised LearningNow that we have a general idea of what machine learning is, how does it actually work?In order to accomplish machine learning, it is essential to teach the machine. There are 2 primary ways to teach a machine — via supervised learning or unsupervised learning. In both learning techniques, a computer is given a set of training data. Supervised machine learning is the more common between the two, and the key difference is that the training data is given labels. Essentially, the data scientist tells the computer what it should think. It’s similar to the way that a child might learn math from a teacher. In this training style, the possible outputs are already known, and the data used to train the algorithm is already labeled with the correct answers. Results from supervised learning can take many forms, but 2 common types are decision trees (categorization) or regression.(L): A decision tree machine learning algorithm that classifies/predicts data based on its known attributes (R): An example of a regression model, in which the best fit equation is used to predict values for data pointsWhile a supervised algorithm learns to ascribe inputted labels to new data, its unsupervised counterpart will look at inherent similarities between the data points and group them accordingly. It relies on clusters. In the example below machine learning* takes the inputted data, and categorizes people based on the number of pets that they have owned:*This example is purely to demonstrate the concept of clusters. The was no actual machine learning involved.Machine Learning in MedicineMachine learning can be a heavily debated topic when applied to the field of medicine. There are many philosophical debates of whether a machine can and/or should replace the role of a physician. That is not the goal of this blog post. Rather, I would like to focus on 2 awesome healthcare applications of machine learning.Autism & BeyondAutism & Beyond is an Apple ResearchKit app, launched by Duke University in 2015, to screen for autism in children, ages 1 to 6. It harnesses the power of the iPhone and uses the self-facing camera to automatically detect the toddler’s facial expressions, analyze their microreactions, and then indicate if there is a potential risk for autism.Autism & Beyond is not a diagnostic tool. Rather, the team envisions it as a possible screening tool for autism or other developmental disorders of children. According, to Guillermo Sapiro, Ph.D., a professor of electrical and computer engineering, Autism & Beyond’s goal is to “develop a screening, like hearing or eyesight at schools…They don’t get glasses; they get a referral.”In order to detect children’s emotions, Sapiro’s team uses facial landmarks to categorize the child’s facial expression to one of 7 categories: anger, disgust, fear, happy, sad, surprise, or neutral.Source: Screenshot from https://www.youtube.com/watch?time_continue=1029&v=S8dqqAoRNWMThe facial landmarks are able to account for pose-variance, through data transformations. Sapiro’s lab’s research heavily focuses on facial recognition through a dictionary learning framework. The image below (on the left) shows a 3D textured face scan from the BU3DFE dataset (which contains 100 subjects with 2500 facial expression models). Sapiro’s team used 19 facial landmarks(highlighted in green) to decompose the sample into a 3D and multiple 2D representations. All of these representations are stored in the “dictionary” (image on the right).Source: http://people.duke.edu/~qq3/pub/expr_final.pdfThe Autism & Beyond study has been completed, and findings have not yet been published. Whether the findings are statistically significant or not, I believe the Autism & Beyond app has already positively impacted our healthcare system. By partnering with Apple and creating a ResearchKit app, Autism & Beyond allowed everyone with an iPhone to have access to the study. Traditionally, clinical trials take place at research hospitals (usually in major cities) and require many in-person visits. This can be geographically and financially prohibitive. To me, this is an exciting breakthrough in healthcare, and the Autism & Beyond seems to feel the same way:“We’re not going to cure autism,” Bloomfield concludes. “But we can help put the structure in place to help manage it and move [treatment] forward. And that’s why we’re all so passionate about this.” -Ricky Bloomfield, pediatrician and the head of mobile technology strategy for Duke Medical CenterTo learn more about Autism & Beyond , see here.AliveCorAliveCor is a healthcare tech company that has created an FDA-approved mobile app that can take a realtime EKG, using either the KardiaMobile electrode sensor or KardiaBand, compatible with the Apple Watch. EKGs are near and dear to my heart (before Flatiron, I spent 4 years as a medical device rep in electrophysiology).Source: https://store.alivecor.com/products/kardiamobileEKGs measure the electrical activity of the heart, and the purpose of KardiaMobile is to detect irregular, abnormal heart rhythms.AliveCor uses a proprietary rhythm monitoring algorithm called SmartRhythm, which utilizes an autoregressive, deep neural network. It learns the normal relationship between heart rate and activity, and notifies the user when it sees an unexpected pattern. Unlike the facial recognition in Austim and Beyond, SmartRhythm uses unsupervised learning to learn for itself what a normal heart rate and activity pattern looks like.This system works by looking at the most recent 5 minutes of activity data (1), and based on the trained model of normal heart rate and activity patterns as well as your historical heart rate (2) and activity data (3), and makes a prediction for what it expects your most recent 5 minutes of heart values to be.The model is well-adapted to different activity levels. In the following examples, we can see AliveCor in action. In the image below, in an ordinary day, with low amounts of activity, we don’t expect to see much variance in heart rate. Any heart rate deviations are small and gradual.While exercising, we do expect to see heart rate changes, and these tend to take the form of what’s called a Gaussian Mixture Model (GMM). More simply, we expect the heart rate to increase with activity, plateau at our peak heart rate, and decrease when activity ceases.Irregular heart rhythms, or arrhythmias, don’t follow these rules. Heart range changes are dramatic and abrupt. In the diagram below, you can see where SmartRhythm has first detected the arrhythmia with the red vertical line.I think AliveCor is awesome because it give patients the agency to provide more information to their physicians to supplement their symptoms. Arrhythmias are unpredictable, sometimes of short duration, and the current options to document them all have their pitfalls:EKG in hospital or doctor’s office — Pitfall: What if the patient isn’t in the arrhythmia when they arrive at the doctor’s office?Holter Monitor (24 or 48 hr ambulatory EKG) — Pitfall: What if the arrhythmia doesn’t happen in time wearing the holter?Implantable loop recorder (implantable EKG monitoring device) — Pitfalls: 1) Requires minor surgery, 2) Device is programmed to record certain heart change changesWith AliveCor, patients can record a realtime EKG, without requiring professional help. AliveCor provides a great overview on its website, which can be found here.Some Final Thoughts on Machine Learning in MedicineIn my research of machine learning applications in medicine, I think I was drawn to both Autism & Beyond and AliveCor because they both use technology to make healthcare more accessible. While we have endless technology at our fingertips, it can often seem that none of it helps us navigate the complex healthcare system. Autism & Beyond and AliveCor both aim to bridge that gap and give patients more power. This is the type of technology that really excites me, and I can’t wait to see how machine learning progresses in our healthcare system.
According to inventor of the World Wide Web and Director of W3C Tim Berners-Lee, “The power of the Web is in its universality. Access by everyone regardless of disability is an essential aspect.” Considering the unparalleled flexibility and virtually limitless possibilities of digital media and technology, there is no excuse for an internet that does not accommodate the full range of human diversity. Over 20% of the population has some kind of disability that might impact their ability to access the web through ‘conventional’ digital platforms. The W3C Web Accessibility Initiative defines disabilities that affect an individual’s ability to access to the web as including auditory, cognitive, neurological, physical, speech, and visual disabilities. It is impossible for those of us who do not have a disability to truly appreciate the barriers that users with disabilities are confronted by when using the internet. I believe that, as web developers, we have an ethical responsibility to identify and eliminate accessibility barriers in the software we write.To understand how to code accessible websites, it is essential to learn about the techniques that users with disabilities already utilize to navigate the web. W3C divides these techniques into the categories of assistive technologies (software and hardware that is designed to facilitate interaction with the web) and adaptive strategies (ways in which users with disabilities utilize mainstream software and hardware to improve accessibility). Oftentimes, the success of assistive technologies and adaptive techniques is contingent on the cooperation of software developers in writing code that is compatible with those accessibility tools. For example, a screen reader is an assistive technology utilized by blind users and users with print-reading disabilities that processes content in a web browser and converts it to text-to-speech or Braille. For screen readers to work, it is essential that a language is specified in the html tags, that the semantic structure of a webpage is accurate and descriptive, and that the content makes sense presented linearly and in the absence of visual design clues. If a language is not specified, text-to-speech software might dictate the text in the wrong language or translate the text improperly. Any pictures used in a website (especially those essential to understanding the site’s content) should be given a descriptive alt tag so they can be interpreted by screen readers. The experience of navigating a website using a screen reader is similar to that of navigating through the menu of an automated telephone system, and we all know how frustrating that can be. Proper html semantics, labels on forms and descriptively named links are essential for users of screen-readers to navigate through a website.Using a refreshable braille display to browse the webAn example of an adaptive technique utilized by people with disabilities for web browsing is keyboard navigation. According to a 2012 survey, 8.2% of Americans have difficulty lifting or grasping, limiting their ability to use a mouse. Most websites are designed with only mouse users in mind, resulting in an accessibility barrier for individuals with mobility disabilities. If a website is designed properly and with accessibility standards in mind, it should be easily navigated using only a keyboard. Individuals using a keyboard to navigate use the arrow keys to scroll up and down a webpage and tab and shift+tab to navigate back and forth between links. For the tab navigation to work properly, links should have some kind of highlight on focus effect turned on so that keyboard users can keep track of where they are on the site. Additionally, links in navigation bars should be ordered logically from left to right and then top to bottom so that the page has a natural flow for keyboard users.Overall, it takes relatively little effort for web developers to make their sites accessible to all users. HTML was designed to be accessible, and using HTML properly means building accessible websites. Considering the increasingly central role the internet plays in our society, it is essential that we cooperate with existing accessibility standards and innovate new ways to make the web even more accessible. Making the web accessible to the full range of human diversity expands the possibilities of future innovation that will benefit us all.Resources for Maximizing AccessibilityAccessibility Developer Tools: A chrome browser extension that adds an accessibility checker to the audit section of chrome developer toolsWeb Developer: A chrome browser extension that has a validate accessibility featureChromeVox: A screen reader browser extension for Chrome. Useful for understanding what it is like to navigate the web using a screen reader.Colorblind Webpage Filter: a web application that simulates how any website would appear to users with various forms of colorblindnessAccessibility Color Wheel: a tool that helps choose a combination of background color and text color that is readable to colorblind usersMicrosoft’s Inclusive Tool Kit Manual: A pdf detailing inclusive design principles and strategies.W3C Web Accessibility Initiative Website: Strategies, guidelines, resources to make the Web accessible to people with disabilities.
Coming from a Cognitive Linguistics into the world of software engineering in general and object oriented programming in particular has its own benefits. Both worlds rely on conceptualization constructs which although having different implementation and context, possess common properties. At their core they provide “a way of characterizing the structured encyclopedic knowledge.”  And while in cognitive linguistics we deal with frames, mental spaces, idealized cognitive models, and semantic fields to name a few, the focus of this particular piece will be a domain model as applied to software engineering.Defining a Domain ModelWhat is a domain model? Here are some definitions that are used in articles and research papers on DDD (Domain Driven Design):“The Domain Model is your organized and structured knowledge of the problem. The Domain Model should represent the vocabulary and key concepts of the problem domain and it should identify the relationships among all of the entities within the scope of the domain.” “A domain model is a visual representation of conceptual classes or real-world objects in a domain of interest.” ‘A visual dictionary of abstractions.”“A Domain Model creates a web of interconnected objects, where each object represents some meaningful individual, whether as large as a corporation or as small as a single line on an order form.” “… a conceptual model of the domain that incorporates both behavior and data.” We may even come up with our own mega definition:Domain model is a structured visual representation of interconnected concepts or real-world objects that incorporates vocabulary, key concepts, behavior, and relationships of all of its entities.The key word in the above definition for me personally is visual. My brain performs at a maximum if a theoretical concept can be visualized in a digestible and somewhat eye-pleasing manner. Most of the time a domain model is illustrated with a set of class diagrams which may show:domain objects or conceptual classesassociations between conceptual classesattributes of conceptual classesBuilding a Domain ModelLet’s take a look at an example of modeling a particular domain and analyze its components. We will be dealing with a simplified Vehicle domain. Here’s a verbal description that will serve as a base for our model:A vehicle is a mobile machine that transports people or cargo. Typical vehicles include wagons, bicycles, motor vehicles (motorcycles, cars, trucks, buses), railed vehicles (trains, trams), watercraft (ships, boats), aircraft and spacecraft. Land vehicles are classified broadly by what is used to apply steering and drive forces against the ground: wheeled, tracked, railed or skied.Using Noun Phrase Identification to Single Out Conceptual ClassesWe will be applying a useful technique — linguistic analysis: identifying the nouns and noun phrases in textual description of a domain, and considering them as candidate conceptual classes or attributes. Therefore, here’s the list of our class candidates:Machine ,wagons, bicycles, motor vehicles, motorcycles, cars, trucks, buses, railed vehicles, trains, trams, watercraft, ships, boats, aircraft, spacecraft. Land vehicles, wheeled, tracked, railed, skied.Transforming the above list into classes with the corresponding Class_names, this is what our Vehicle domain model may look like:Let’s dive into the structural components of the above domain model. We are using UML (Unified Modeling Language). A concise structural analysis will give us the following results:When we see a straight line connecting 2 classes together this shows us that these 2 classes are related.An arrow implies a relation of inheritance. So for the diagram above, its should be obvious that Land_Vehicles will inherit some methods from Vehicles.The line with a diamond on the end states is that class ‘A’ contains class ‘B’. For example, above we say that a Cars and Trucks contain Wheels.Numbers. This is a case of multiplicity. It defines how many instances of a class A can be associated with one instance of a class B. In other words, what numbers state is a relation such as one-to-one, or one-to-many. In the example above we have these numbers on the ‘contains’ relation for Cars and Trucks and Wheels. We may say that one (1) Car contains four (4) Wheels. And one (1) Trucks contains four or more (4*) Wheels. It is important to note, that we can only use numbers on plain relation lines and contains relations, but NOT on inheritance relations. Sources:Cienki, Alan. (2007) Frames, Idealized Cognitive Models, and Domains. The Oxford Handbook of Cognitive Linguistics 170–187. Oxford: Oxford University Press.Brown, Philip. (11.12.2014) What is the Domain Model in Domain Driven Design? Culttt Retrieved from: https://goo.gl/cLDiTTLarman, Craig. Applying UML and Patterns. An Introduction to Object-Oriented Analysis and Design and the Unified Process. Second edition.Fowler, Martin. (11.11.2009) Framework Design Guidelines: Domain Logic Patterns. Retrieved from: https://goo.gl/mWsL5RWikipedia article on Domain Model.Badgerati.(11.26.2009) What is Domain Modeling? Computer Science Source Retrieved from: https://goo.gl/ti382L
It happens to be so that the we are surrounded with dichotomies. Black vs. white, good vs. evil, Sinatra vs. Rails — the list goes on forever taking its root in philosophical dualism and expanding into our mini conversation on duality in programming in general and Ruby in particular.Let us be clear though that the term dichotomy is not a perfect lexeme to describe the relations between the concepts in our brief exploration. Dichotomy is defined by Merriam-Webster as a division into two especially mutually exclusive or contradictory groups or entities. Therefore for our purposes we will only use the second half of the definition — contradictory groups of entities (or so they may seen to a newcomer into the world of programming). With linguo-philosophical preface behind us, let’s dive in into the world of Ruby dichotomies*.Parameter vs. ArgumentA parameter is a variable in a method definition. When a method is called, the arguments are the data you pass into the method’s parameters.An alliterating mnemonic that may help:PARAMETER → PLACEHOLDERARGUMENT → ACTUAL VALUEDeclaration vs. DefinitionDeclaration: You are declaring that something exists, such as a class, function or variable. You don’t say anything about what that class or function looks like, you just say that it exists.Class declarationDefinition: You define how something is implemented, such as a class, function or variable, i.e. you say what it actually is.Class definitionStatement vs. ExpressionWhile there is no difference between expressions and statements in Ruby due to the fact that everything evaluates to a value and therefore everything is an expression, there is a difference in other languages.An expression is a statement that returns a value.Statements do something and are often composed of expressions.Expression vs. statement in PythonStateless vs. StatefulStateless means there is no memory of the past. Every transaction is performed as if it were being done for the very first time. e.g. RESTful web services are stateless by design.Stateful means that there is memory of the past. Previous transactions are remembered and may affect the current transaction.PATCH vs. PUTPUT replaces the resource in its entirety.PATCH updates part of the resource..new vs. .create.new instantiates an instance that lives in Ruby’s memory and is not saved to database..create instantiates an instance and saves it to database..delete vs. .destroyWhen you invoke delete on an object, ActiveRecord merely tries to run the DELETE FROM tablename WHERE conditions query against the db, performing no other ActiveRecord-level tasks.When you invoke destroy on an ActiveRecord object, the ActiveRecord'destruction' process is initiated, it analyzes the class you're deleting, it determines what it should do for dependencies, runs through validations, etc.In other words:delete will only delete current object record from db but not its associated children records from db.destroy will delete current object record from db and also its associated children record from db.Their use really matters:If your multiple parent objects share common children objects, then calling destroy on specific parent object will delete children objects which are shared among other multiple parents.find vs. find_by.find returns a single record or an array of objects if multiple arguments are given. There is no implied ordering of the return — an array is returned in the order found. If one or more of the arguments are not found, it raises an exception, ActiveRecord::RecordNotFound and returns a 404. To use this method and avoid this error, you can first search for the existence of the id number in your records, but this can be time consuming.Song.find(1) => the song object where id == 1.find_by accepts a key value pair as a parameter by which to search the data structure. If the object is found, it is returned. If it is not, a nil value is returned and your program continues to function without exception. It finds and returns the first match.Song.find_by(id: params[:id])Static vs. Dynamic RoutesStatic route — A static route will render a view that does not change. Typically, you will not send parameters to it. Examples would be a site’s about or contact pages.Dynamic route — Dynamic routes are pages that accept parameters and render different content based on those parameters. An example would be a blog’s post page that contains a specific article.EDD vs. TDDEDD — error driven developmentTDD — test driven developmentExplicit vs. Implicit RenderingExplicit rendering — for explicit rendering, Rails lets you dictate which view file you want to have the controller action mapped to.Implicit rendering — for implicit rendering, Rails follows a standard convention that automatically looks for the view file with the same name as the controller action.form_tag vs. form_for in RoRform_tag simply creates a formform_for creates a form for a model object.They are not interchangeable and you should use the one that is appropriate for a given case.form_for is a bit easier to use for creating forms for a model object because it figures out what url to use and what http method to use depending on whether the object is a new record or a saved record.Do you have your own examples of Ruby dichotomies? Please add them to the comments below.The list is far from being full, therefore this post will be updated periodically.Most definition and comparison analysis is based on answers and comments on stackoverflow.com as well as publications on Medium.
My ultimate goal is to combine my cognitive linguistic background with the new knowledge in software development and dive into the world of NLP. With hundreds of NLP APIs and SoS readily available to be integrated into your apps reaching this goal becomes plausible. Here I will give a quick and rough introduction into the world of NLP.Back in college when I was getting my minor in psychology, NLP was all a rage but it was referring to a completely different thing — Neuro-linguistic Programming. Now the NLP acronym is back in spotlight but with a different meaning attached to it — Natural Language Processing.In a nutshell, NLP helps computers understand human language. And human language is a weird concept. Not only you have syntax, grammar, phonetics which are somewhat straightforward with digestable set of rules, you also have semantics… and this is a completely different animal. A neural network has to be really smart to understand all stylistic devices like metaphor, hyperbole, metonymy, anaphora with epiphora, synecdoche, etc. NLP is a key component of artificial intelligence (AI) and relies on machine learning, a specific type of AI that analyzes and makes use of patterns in data to improve a program’s understanding of speech. We can take Scott Simms’ categorization of common uses of NLP as a base and build on it:Common uses of NLPSentence segmentation (tokenization), stemming (create a word stem for each token), stop words removal, part-of-speech tagging. Hence, NLP is used to analyze parts of a sentence to better understand the grammatical construction of the sentence. Libraries: NLP Toolkit, NodeNatural.Deep analytics. Involves the application of advanced data processing techniques in order to extract specific information from large or multi-source data sets. Deep analytics is particularly useful when dealing with precisely targeted or highly complex queries with unstructured and semi-structured data. Libraries: Watson Discovery,Machine translation. One human language is automatically translated into another human language. Tools: Google Translate, Microsoft Translator.Text-to-speech and speech-to-text. Convert audio and voice into written text and text into natural-sounding audio. APIs: Watson Text-to-Speech, Watson Speech-to-Text.Natural conversations. Build conversational interfaces into your applications (chatbots). APIs: Amazon Lex, Watson Conversation.Automatic text summarization. Produce a readable summary from a large chunk of text. Document categorization.Content classification. Understand the overall sentiment expressed in a block of text. APIs: Google Cloud Natural LanguageSentiment analysis. Understand the overall sentiment expressed in a block of text. APIs: Google Cloud Natural Language, Watson Personality Insights, Watson Tone Analizer.Semantic analysis. Contextual intelligence, audience profiling and segmentation. APIs: Leiki.In a word NLP helps computers to help us to make sense of what we say.‘’If I loved you less, I might be able to talk about it more”This is just a gist of all the fun things that NLP provides. Next step is building apps using the above mentioned libraries, APIs, and services.
Group paring and TeamworkBefore joining Flatiron I was coding on my own without any other human interactions, and until recently I thought that that is how it’s supposed to be done, everybody does their own work and afterwards combines it into something bigger. But boy have I been misinformed. This blog will cover my, recently developed, point of view on group paring and teamwork.At the end of module one we had a whole week to work on a project in pairs of two. And to be honest at the end of the project I felt really proud of the program that my team-mate and I managed to develop. Although I think the whole thing would’ve started better if I knew, from the beginning, the things that I’ve learned near the end. I’m going to list some useful things to pay attention to while working with others.“Inform your teammate on what you’re doing and what’s the code you’re typing going to do.”To be honest this one was driving me crazy and I’ve caught myself doing it a few times. What tended to happen is when presented with a problem my team mate and I would discuss it and all of a sudden I would get an idea on how to solve it. At that point all interactions with my teammate would stop, and I would start to ‘furiously’ type code that made sense to only me blocking all questions from my teammate, which would leave my teammate stunned for however long it took me to complete my thought.And I feel that this was happening because I was afraid that I would loose the thought and with that also the solution that was in my mind. After I put some thought into this issue, I came up with a solution. Every time I would get an idea I would write it down and discuss it with my partner and make sure he/she understood it before implementing it. Which sometimes ended up in my teammate seeing a flaw in my code that I otherwise wouldn’t notice and then working together to fix that flaw and actually getting the code to work. The point is SHARE YOUR VISION with your teammate, it might become even better that you’ve first thought.“Equal workload, getting to know the strengths and weaknesses”This one is more important than the first one. It requires that you know what you’re good at and what you’re not, and then discussing that with your partner. Because when you do you’ll be able to cover much more in a smaller amount of time. Let’s say one teammate is better at designing and creating a visualization of your code while the second one is better at creating methods and the logic behind them. You need to let each other know so you can focus on those specifics. And also when a problem occurs that you and your partner are not so familiar with you can work together on it and learn. On the other note it’s very important to have patience and to be willing to explain to your partner the things you have more understanding in, cover what you wrote until everybody is on the same page and everything is as clear as it can be. Be positive and don’t beat each other up over mistakes, because those mistakes are going to make you learn more. Also just because one person is typing doesn’t mean you’re not contributing, so don’t feel bad or as if you aren’t doing any work. Sometimes all it takes is a single sentence, or even a word, to make an idea appear within one of your teammates, and that is a form of contribution you’re not giving yourself enough credit for.“Finding a way to work even when you’re not at the same place”Some times you’ll find yourself in a position where you won’t be able to come to class/work etc. and will have to stay home, and let me tell you that that is not a reason to ditch a teammate and do nothing all day (unless you’re very sick or something else bad happened, in which case you have the full right do whatever you need to). These days you have so many ways to communicate with each other over the internet that I can’t even name them all. Now to get to my story. After my teammate and I finished our project we headed home for some well deserved rest. But after returning home we figured that our code had many things that could’ve used refactoring (unnecessary lines, lines that we could’ve put inside of methods to cut unnecessary repetition, and some lines that were never even being called), but before we started working on it by our selves, we contacted each other to bring up the idea that we can make this code look much better and user friendly, because we didn’t want to make any changes that would leave one of us in the dark or even worse break our code the day before presentation. We set up ‘Zoom’ which is an app that has voice and video chat and also allows screen sharing and for the user to control the screen of the other user, which means that if one of us didn’t understand the other, we could simply type it out on their screen where they are typing the code. By using this app and communicating we ended up making our code look cleaner that we planned and we managed to cut almost 100 lines from our code and still retain the original functionality, all from the comfort of our own home.To end this off I would like to recap on the important keywords which are: talk, listen, work, rest, patience, and the most important one, teamwork!
Today i want to talk about atom packages that won’t just make atom look better but also give it tons of new and useful functionalities.EmmetThe number one code snippet tool used by front end developers. Emmet helps to create HTML and CSS faster with the use of snippets. It uses abbreviations that expand to valid HTML tags.2. BeautifyThe atom-beautify package will clean up your code, and make it more readable.Before BeautifyAfter Beautify3. Ask StackThis package i found very useful since it lets you search stack overflow straight from atom4. Git PlusThis package provides a bunch of shortcuts to commonly used git actions, without the need to switch to terminal. You can bring up the commands using cmd + shift + D5. File IconsThis package displays ‘better’ icons next to files in your file-tree. They help to visually parse file types with little effort. You can choose between colored or monochrome icons.
Internet art, also called net.art (named after the result of a software glitch, 1995, read more here) describes work made in the 1990s through the early 2000s that delivers an aesthetic experience through using the Internet. It is often but not always participatory and multimedia based.In 1952, the computer scientist Christopher Strachey developed a program for the Manchester Mark 1 computer that created randomized love letters. It is credited as the first example of computer art. It wasn’t until the internet was accessible to more people in the mid 90’s that net.art started to develop. (Originating from Eastern Europe and Russia, read more here.)Below are a few of the most influential and important internet art works from this time.It is difficult to see some of these now because a lot of this work has been lost or poorly archived. This is a common trend due to the internet and technology’s constant updates. It is sad but almost part of the net.art aesthetic for these works to fade away as progress marches forward.They’re are a few resources that I was able to find that are pulling these programs/websites out of obscurity and displaying them. (links at the bottom). The best that I’ve found is Rhizome, which was one of the first platforms to share this kind of art in the 90’s and is still functioning today. They just started a project where they will release important net.art over a period of 2 years. The coolest part of their display is that they somehow are able to open a windows 95/98 session or use a netscape browser within your browser. That to me is really interesting in itself. Here’s an example: http://archive.rhizome.org/anthology/epithelia.htmlJodi.org: (1996) A collaboration between Dirk Paesmans and Joan Heemskerk, acquired cult status for its ability to jolt viewers out of routine web surfing. Visiting the site would crash your computer. The website causes fluttering browser windows to rapidly open all over your screen, each triggering automatic downloads that duplicate the process every few seconds, a process that goes on until your browser crashes. Restarting your computer just makes it happen again and again. There is a way to turn it off, but it requires getting into the website’s code. Jodi.org went on to create all kinds of annoying web projects. The website still exists without crashing your computer, so it’s possible to view.My boyfriend came back from war:(1996)Created by Olia Lialin, is an example of interactive hypertext storytelling. The work consists of nested frames with black and white web pages and grainy GIF images. When clicking hyperlinks in the work, the frame splits into smaller frames and the user reveals a nonlinear story about a couple that is reunited after a nameless military conflict. The woman confesses that she has had an affair with a neighbor while the returned soldier proposes marriage.Form Art: (1997) Created by Alexei Shulgin, is an interactive website navigated aimlessly through a series of web forms. Forms’ are HTML conventions that appear as menus, checkboxes, text boxes and labels. Forms have a conceptual appeal since their content is often submitted seemingly into the ether. On this website, viewers are invited to explore a labyrinth accessed through forms and hyperlinks. Forms lead to other larger combinations of forms, pointlessly.Data Diaries (2003) Created by Cory Arcangel, is an abstract autobiographical work using the contents of his computer’s temporary storage (RAM) as raw material for a series of abstract, glitchy videos. Each day over the course of a month, he would employ a hack to open this normally unseen data in QuickTime, which translated it into frenetic patterns of blocks and digital static.“The pure shapes and tones of real computer memory. Each video documents a new day, and each day the computer offers us a new set of memories”, albeit illegible. Arcangel is more famous for his later work where he displays scrolling Super Mario clouds. (not technically net.art)World of Awe: The Traveler’s Journal: (2000) Created by Yael Kanarek, visitors access the Traveler’s story by opening their files, reading accounts of their journey and their love letters. When entering this world, the user lands on a simulated computer desktop. Clicking on any of the desktop icons leads to clues as to what they are doing. The Traveler’s laptop is a glimpse into their inner life.The World’s First Collaborative Sentence:(1994) Created by Douglas Davis, allows users to contribute to a never-ending sentence. It is credited for anticipating today’s blog environments and ongoing posts. The Whitney Museum has recently brought this back to life and you can still add to it today. Linked here is the original.Current WorkAmerica’s Got No Talent (2012): Created by Jonah Brucker-Cohen and Katherine Moriwaki, this app processes the virtual presence of several American reality television shows through how popular their Twitter feeds are. The visualization takes the form of a horizontal bar graph in the shape of an American flag that updates dynamically. This creates a measurement meter that ranks popular media on their social exposure, rather than their credit as viable media sources. (00:50 on youtube)http://intotime.com/: (2010) Created by Rafaël Rozendaal, this is just one of hundreds of websites created by Rozendaal, each of which offers a simple, usually interactive experience that fills the browser window and bears a dedicated domain name as its title. This approach, honed in the early 2000s as part of the “Neen” movement, has allowed Rozendaal to compile study after study of the browser’s potential for aesthetics.http://www.floatbounce.com/http://www.trashloop.com/http://www.lotsofmany.com/Every Non-Word(2015): Created by Allison Parrish, attempts to exhaust all possibilities of non-existent English words. Each non-word is a randomized collection of syllables, first collected by using an API of the entire English dictionary. Each resulting word is checked against a dictionary to rule out accidental real words. Twitter bots are a huge thing. See also @Horse_ebooksScreendust:(2016) Created by Protey Temen and Zakhar Day, this simulation is based on a contingence algorithm that creates particles with a discrete structure and is divided into segments. Each millisecond, a recalculation of values occurs.Goodmorning Goodnight:(2015) explores visual and textual representations of sunset and sunrise in the online environment. Overlaid on a grid of latitudes and longitudes are location-specific images of sunsets and sunrises taken from Panoramio, a photo sharing website. Superimposed over the sunrise and sunset images is a layer of text comprised of scrolling lines and comments scraped from livedash, a website that allows users to search for particular words or phrases on national television. In JODI’s signature style, the web is turned inside-out.(00:24 on youtube to see it)Resources for more inspiration:http://thewrong.org/http://artport.whitney.org/gatepages/index.shtmlhttp://turbulence.org/https://anthology.rhizome.org/
How computers and databases mingleSQL stands for Structured Query Language. SQL is a programming language that we use in order to access data from within databases.Ruby talks to a Database by using SQL as the languageBrief historical noteSQL was developed around the 1970s by the late Edgar F Codd. (However credit is to be given to Donald D. Chamberlin and Raymond F. Boyce for helping create and develop SQL, all colleagues at IBM)Codd was an English Immigrant who had come to the United States after WWII, to work at IBM as a mathematical programmer. It was here during his time at IBM where he began to develop and iterate on the idea of Relational Models the cornerstone to SQL.Relational ModelsIs the idea that data can me modeled and grouped by relationship. It is a collection of data, that are all connected with one another by a key shared amongst them.For a more in-depth understanding definitely check out the Wikipedia page, as well as some computer science courses.https://en.wikipedia.org/wiki/Relational_modelWhat is a relational database ?The classical organization of data into a structured format. Akin to a phone book — these consist of two or more tables with columns and rows.Every row represents an entry in that tableEvery column represents a data typebasic table setupIn order for these to be their most effective, they require instructions for storing data within them. Just as we don’t randomly throw phone numbers into a phone book we don’t do the same for Relational Databases.Schema — the relationship between tables and column data typesPopular SQL DatabasesMySQL, Oracle, IM DB2, Sybase, MS SQL Server, Microsoft Azure, MariaDB, PostgreSQLEach of these tables shares a similar point of dataWhat is a non-relational database?The organization of data that is unstructured by nature. Akin to file/folders of yonder — they assemble related information across multiple databases. Non-relational databases are more commonly referred to as NoSQL databases. (Not Only SQL)Popular NoSQL DatabasesMongoDB, Apache’s CouchDB, HBase, Oracle NoSQL, Apache’s Cassandra DB, RiakWhat’s the difference?As defined earlier, SQL is a structured, rigid language that demands that prior to data coming in that it adheres to a schema — aka a outline for how the data is stored. Data has a clear and defined pathway and link with other data in the database.SQL is for smaller datasets, whereas NoSQL performs better with larger datasetsSchema’s defined within some simple SQL databasesNoSQL Databases are different in that the databases are setup more like a conventional computer with documents. We group a collection of things together and store them next to each other to reference at a later point — Imagine grouping blogs, videos, images, and gifs that all pertain to rabbits, within a folder for rabbits — its just a bit more intuitive, then creating tables with defined molds of what is allowed. NoSQL can be seen as more flexible alternative to data storage.Different Types of Non-Relational DatabasesKey-Value ModelStores data with indexed keys and valuesColumn StoreStores data tables in a column/row configurationDocument DatabaseA more complex version of the Key-Value model, wherein each document has it’s own unique key & data.Graph DatabaseFor storing interconnected data that can be best displayed using graphs — very complex stuff can be had!
*not only SQLWhat is it?A database type — that stores data in a non-relational fashion. This means instead of the data being stored in the classical sense of tables — columns and rows, the data is stored in a document or object fashion. What does that mean? The data is more akin to a Ruby Object than a set of related rows and columns. Think of traditional SQL being an Object that gets split amongst varying tables and NoSQL being an object that retains itself.Most commonly associated with Big Data and real-time web-applications.Schema-freeSimple & fast horizontal scalingHandles large data easierSupports standard SQLWhy is it a thing?NoSQL saw a surge of popularity in the early 2000s as big tech companies like Amazon, Facebook & Google ran into problems with their data storage and the limitations of standard relational databases. As users of the web began to increase in volume and the concept of Web 2.0 became more apparent NoSQL filled a gap that such companies were looking for. The ability to access data fast, to scale that data fast and operate on that data.NoSQL has no pre-defined schemas, the turn-around time from instantiation is quick. An essential feature of Web 2.0, or ‘new age’ web applications and websites.Growth of internet usage 96' — 18'But what exactly is Web 2.0 and what does this have to do with databases?Web 2.0 is a term that describes the emphasis over the last 15 years in user-generate content, usability, interoperability*. The rise of websites like Reddit, Facebook, & Youtube encapsulate the featurettes of Web 2.0 Recommend you read the wikipedia article on Web 2.0* the ease-ability of different websites working in tandemWith these kinds of websites, came with it a demand for faster, and larger data storage systems that could handle the workload of billions of users and millions of requests per second! This is what ‘Big Data’ is — a lot of data!Graph showing the eventual performance drop-off of Relational DatabasesData that is both structured and unstructured and is frickin’ massive in volumeTiny Historical NoteThe actual term, NoSQL was coined initially by Carlo Strozzi in 1998. But has it’s roots from the 1960s when the exploration of database technologies was on the rise.What are some kinds of NoSQL database systems?There are a slew of variants of non-relational NoSQL databases. Below I have*Note: Some of these database models are hard to abstractly draw or show representation of— but they model OOM very closelyVarious types of Databases available to be usedThe 5 most commonly associated models are as follows:1) ColumnThe column is used as a store for the value, each come equipped with a timestamp to individuality (unique attribute). Good for analytics, and is reasonably quick.Database examples — Vertica, Cassandra, Druid, HBaseExample of a Column’s actual data (in a JSON-like notation)2) DocumentUnlike a traditional SQL table where an object’s data is stored across multiple tables, document databases store all information for an object in a single instance of the database. The key factor for Document stores is that they are similar with one another, but allow for different types of documents within an instance and allow the fields within them to be option. Great all-around option (complexion/size)Database examples — Apache CouchDB, MongoDB,RethinkDB, BaseXExample of a Document within a Document StoreThey are essentially the same thing as an Object in OOM programming in Ruby.3) Key-ValueDesigned mainly for storing associative arrays otherwise known as hashes. Great for very large data models.For example we can have the following hash be represented by the key customer8572 which points to the following hash of data.Database examples — Apache Ignite, Couchbase, OrientDB, Redis, OracleExample of a hash value assigned to a key4) GraphHave their basis in Graph Theory — but employ the idea of nodes, edges and properties. They can be though of in the context of Ruby models, where one Model has a relation to another. Great for more complex data models.Database examples — ArangoDB, Apache Giraph, OrientD, MarkLogicExample of a Graph DatabaseNodes — Are entities, think of people, items, business accounts. *Anything that can be trackedEdges — Sometimes referred to as relationships, are the connection between nodesProperties—Relevant information to nodes5) Multi-ModelThese are database models that incorporate many of the aforementioned models into a single database. Think of it as the grouping of multiple database components into a single powerhouse.Database examples — Apache Ignite, Couchbase, OrientDB, ArangoDBAn example of multiple database variants working in unisonWhile powerful, each database system has it’s strengths and weaknesses and this is particularly important to remember when building out applications and services. There is no one-size-fits-all solution to data storage, there is however a tactical approach you can take to optimize systems and increase efficiency. By knowing how some of the systems work at their core you can have a better understanding on the whole of how systems operate in a holistic picture.MongoDBAs part of the Flatiron Immersive Web Development, one of the database technologies taught is MongoDB. In my next blog I will take a look at MongoDB.https://www.mongodb.com/Sourceshttps://en.wikipedia.org/wiki/Global_Internet_usagehttps://en.wikipedia.org/wiki/Web_2.0https://www.w3resource.com/mongodb/nosql.phphttps://en.wikipedia.org/wiki/NoSQLhttp://udbms.cs.helsinki.fi/?projects/ubenchhttp://blog.datatons.com/2017/06/08/nosql-relational-databases/