coding101

Why This, Why Now…

22/05/2016 By debkr

goal = 'career-change'
reason = 'https://twitter.com/Qwiery/status/727849124138192896'

# PTP: Programming Stream (url = 'http://deborahroberts.info/2016/02/diving-into-data-syllabus-2/')
longlist = ['Java', 'Python', 'SQL', 'VBA', 'R']
start_with_end_in_mind = {'datascience': ['R', 'Python'], 'machinelearning': ['Python', 'R']}
choice1 = start_with_end_in_mind.get('datascience')
choice2 = start_with_end_in_mind.get('machinelearning')

shortlist = list()
for item in choice2 :
    if item in choice1  :  shortlist.append(item)

quit()

# Due to unorthodoxy I executed the last line just after line 02

https://twitter.com/Qwiery/status/…

http://deborahroberts.info/2016/02/diving-into-data-syllabus-2/

Coding 101 (part 11) XML and data serialisation

22/05/2016 By debkr

Quick recap:
In part 10 of this series I learnt a bit about using both the socket library and the urllib library to browse a web page or some other file on a web server, read it and return it as a text file (including HTML tagging where this was included). I put together two little programs that help me to (a) scrape data or a web page from the ‘net (based on a specified URL) and save it to a text file, and (b) handle the most common HTML tags in that text file. The tags handled so far are as follows:

<h1>..</h1> tags: cleaned and saved, labelled as ‘Title’;
<h2>..</h2> tags: cleaned and saved, labelled as ‘Header’;
<h3>..</h3> to <h6>..</h6> tags: cleaned and saved, labelled as ‘Sub-header’;
<em>..</em> tags (italics): cleaned and saved, labelled as ‘Para-header’ (Paragraph header);
<p>..</p> tags: indicate text paragraphs, cleaned and saved only (no additional labels added);
all other tags: ignored.

Coding 101 (part 10) More on Web Data

09/05/2016 By debkr

Python gets networked:
Analysing data from files we already hold on the hard-drive is great, but so much data’s being created out there on the internet (especially on social media websites) that we can use for a whole variety of purposes – I’m itching to get my hands on some web data to play with. First I need to learn about how web browsers talk to websites – that is, how my query (view this website url, download that document, search for such-and-such a search term) gets communicated across the ‘net, and how it gets translated into an instruction the website at the other end can understand (in whatever server-side language it might be using: PHP, JavaScript, or whatever). [Read more…]

Coding 101 (part 9) Python and the Web

07/05/2016 By debkr

Python and the internet data playground:
The internet is a giant data playground just waiting for us to explore it. This part of the Coursera Course I’m studying (see refs below) covers how to collect data from the web so we can easily record, manipulate and analyse it.

This is where the increasingly-common terms web scraping and parsing come in. (Scraping refers to collecting data from the ‘net, while parsing refers to reading and analysing strings of data/info from the web, just like our previous examples of reading text from files.)

We’ll get to access data using web APIs (Application Programming Interfaces), and learn how to handle data in different technical formats like HTML, XML and JSON. So this is definitely where things will start to get exciting. [Read more…] about Coding 101 (part 9) Python and the Web

Simple Tagging Engine (updated using tuples)

07/05/2016 By debkr

My simple Python Tagging Engine has now been updated to incorporate learnings about tuples, and incorporates simplified ways of sorting and returning counted values from a dictionary. Code and simple documentation available on GitHub: https://github.com/debkr/blog_tagger.

Coding 101 (part 8)

03/05/2016 By debkr

A quick recap:
We’re cracking on well here, going through the Python course on Coursera as taught by Dr. Chuck (see footnotes for more info and links). We’ve looked a strings and how we can slice, dice and extract data from them using find and split functions. We’ve gone on to learn about files, and proceeded to open and read data from files, both as strings and as lists. We’ve been able to index lists using integer values from zero upwards. Then we’ve gone on to look at dictionaries, which are mini two-field databases of key/value pairs referenced using their keys. And all throughout we’ve been learning about and using various kinds of loops or iterations, conditional statements, and functions (both built-in and defined in-program).

And now for something completely different:
All these things were familiar to me from programming in school and as a maths undergrad (albeit a little rusty!). But now we come on to something completely different, something I hadn’t heard of before: tuples. I’ll call tuples the big brother of lists as they’re very similar to lists: they’re basically another type of collection of things. [Read more…] about Coding 101 (part 8)

Coding 101 (part 7)

29/04/2016 By debkr

Lists work great but they leave something on the table:
I’ve been building a Tagging Engine in Python as a little exercise to help me learn by doing, using my knowledge so far. It became clear pretty quickly that I needed a better way to handle pairs of data. In this case I was looking at a list of words and the number of times each of them appeared in a text, so that I could rank the most common words by order of significance (frequency). If I just used one list and appended both the word and its count to the list, one value after the other, there was no way I could sort by count number.

I got round this problem by having two lists, one for the words and another for the word counts. I could then manipulate the data as needed. This did work fine in the simple program I wrote, but it was my usual unwieldy, sledgehammer approach again. I knew there was a way I could handle that pair of data points better – using Python’s Dictionaries functionality – but I didn’t want to rush ahead of the curve. Well now I get the chance to learn all about dictionaries. [Read more…] about Coding 101 (part 7)

« Previous Page