debkr – Page 6

Machine Learning Engineering

19/07/2016 By debkr

Dipping a toe in the Udacity pool:
I received an email from the UK/Europe Business Growth Manager of Udacity, looking for people with knowledge of/experiencein Machine Learning to lead and/or support face-to-face training sessions in Machine Learning in London. The email included the link to Udacity’s Nano Degree in Machine Learning.

While I’m not yet skilled or experienced enough at this time to lead a session, I felt really excited that I might have an opportunity to be a session supporter and assistant. That would be cool as I’d be exploring and networking in such an important growth area in tech skills, as well as supporting the growth of MOOCs here in London, which is something I feel really passionately about. [Read more…] about Machine Learning Engineering

Mathematics Mondays (revising some basics)

18/07/2016 By debkr

Here’s a quick revision and recap on some basic maths and linear algebra needed when studying Machine Learning. This will covers basics of calculus and linear algebra. A future post will revise some basic statistics, then I’ll go on todo some quick example calculations of standard deviations, derivatives and matrix determinants.

1. Linear Algebra – matrix multiplication:
Matrices can basically be though of as arrays of numbers, which might be a simple array (equating to a list in Python) which can be expressed as a simple one-row or one-column matrix:

, or they might be more complex matrices with multiple rows or columns: [Read more…] about Mathematics Mondays (revising some basics)

D3.js and Data Visualisation

11/07/2016 By debkr

Data analysis process:
When we encountered the data analysis process earlier in the year, we saw the basic process consists of: gather; clean; analyse (including, checking for accuracy); and finally, visualise/present. We’ve been doing lots of Python programming coupled with creating SQL databases to extract data from some source (web pages, files, XML or JSON files) and sort or store it in a database.

The process we’ve been using during the capstone course – and in line with the original Page/Brin search engine process – is to first collect the raw data and store it – unprocessed – into a holding database. From here we’ve gone on to clean up the data and save it in a more structured way in a new, relational database. This results in a smaller database which is quicker to search and retrieve data from. As I found when writing my own search engine application, these first two databases take a long time to retrieve the data, especially when the search engine’s reach is set widely. [Read more…] about D3.js and Data Visualisation

Simple Search Engine in Python

28/06/2016 By debkr

Part of the Python specialisation capstone (see Refs below) is to recreate a simple web search engine, modelled on the original Google search ranking algorithm (you can read the short version of Page and Brin’s 1998 Stanford paper here). The Google algorithm placed emphasis on information obtained from the HTML “link structure and link text” of all links found in all indexed web pages, and to use this information “for making relevance judgments and quality filtering”.

Google search algorithm:
The basic premise of the algorithm is a probability measure, expressed in laymen’s terms as: “how likely is it that a random surfer would alight on this particular web page if they just randomly surfed through all links on all pages on the web until they got bored and gave up”. The algorithm itself includes a measure of all incoming links to a web page (i.e. the number of “citations or backlinks” to that page), enhanced by the quality-ranking of each of those in-coming citation links. In this way, the search algorithm defines an objective page rank or search ranking for each web page. [Read more…] about Simple Search Engine in Python

Mathematics Monday: Simple linear regression (revision)

13/06/2016 By debkr

Mathematical compass:
Someone recently suggested I make mathematics my compass (while I was busy diving down rabbit holes reading up on twistors and the like). Maybe they’re correct. I realised that if I was going to do that, I’d need to brush up on my map-reading skills first!

I’ve been so busy engrossed in learning Python I’ve sorely neglected the other streams of the training syllabus I set myself back in late February (is it really that long ago?). So I decided I need to dedicate at least one day a week to bringing my maths back up to speed.

I’m having to start almost back at square one. (Demoralising.) Calculus is a tad rusty to say the least: serious revision required. Did I really quit accountancy for this? Appears to be madness. Surely there’s some hidden logic in there somewhere, it just seems pretty elusive right now. Well, onwards and upwards anyway… [Read more…] about Mathematics Monday: Simple linear regression (revision)

Coding 101 (part 15) Databases and beyond

09/06/2016 By debkr

SQL-databases-with-many-to-many-relationships

Getting even more complex – many-to-many relationships:
We’ve learnt that there are several different kinds of relationships in a database (dependent on what data we’re trying to model). When we map out our data model, we should try to capture each the relationships between each of the tables in the model. In database terminology, the nature of each table-to-table relationship is referred to as the cardinality of the relationship.

Previously we looked at databases with one-to-many (and their converse, many-to-one) relationships using a Primary Key as the unique, auto-incrementing ID number in the One-Table (e.g. Recipe Type) and linking this through to the Foreign Key ID number in the Many-Table (e.g. Recipes). [Read more…] about Coding 101 (part 15) Databases and beyond

Python + SQL: example database

05/06/2016 By debkr

XML + SQL + Python:
Here’s a quick example showing how powerful these elements are when we put them together – we can use Python to read data from an XML file, extract data elements we’re interested in, create an SQL database and upload the various data values into the database. We can then query and return various data selects direct from Python (although still have the option to view/query the database through the SQLite web browser as well).

Here I’ve taken the recipe XML data format and saved as a file, which looks like this: [Read more…] about Python + SQL: example database

« Previous Page