Python for Data Science – Importing CSV, JSON, Excel Using Pandas

Although I think that R is the language for Data Scientists, I still prefer Python to work with data. In this blog post, I will show you how easy to import data from CSV, JSON and Excel files using Pandas libary. Pandas is a Python package designed for doing practical, real world data analysis.

Here is the content of the sample CSV file (test.csv):

name,email
"gokhan","gokhan@gmail.com"
"mike","mike@gmail.com"

Here is the content of (more...)

Starting and Stopping Oracle Reports Servers with WLST

Normally people are using to start and stop their Oracle Reports Servers the by Oracle provided scripts startComponent.sh and stopComponent.sh in the $DOMAIN_HOME/bin.

The problem with this set of scripts is, that they really take long time to complete and you need to execute it for each Reports Server:
cd $DOMAIN_HOME/bin
# Lets measure the time
time ./startComponent.sh rep_server1
Starting system Component rep_server1 ...

Initializing WebLogic Scripting Tool (WLST) ...

Welcome to WebLogic Server (more...)

Parsing Named Arguments into Python Scripts

If you are coming from the Shell Script World, you are used to parse Named Arguments into your Shell Script, e.g "-u" for Username, "-p" for Password and so on. With Python Scripts you normally pass ordered arguments into your Python Script, so you are bond with the arguments in a fixed order, while passing Named Arguments you can reshuffle them in any order.

A typical Shell Script Snippet would look like that:

export  (more...)

Building Classrooms in the Cloud

Jumpbox Lab Server

Let’s face it: education without interaction is about as effective as shouting origami instructions at a lumberjack who is cutting down trees. Sure, your informative lessons will come in handy when the product of their work finally becomes paper, but it will be long forgotten and ultimately worthless by then. The only way a student is going to learn is if they can put (more...)

A Neural Network Scoring Engine in PL/SQL

Topic: In this post, you will find an example of how to build and deploy a basic artificial neural network scoring engine using PL/SQL for recognizing handwritten digits. This post is intended for learning purposes, in particular for Oracle practitioners who want a hands-on introduction to neural networks.


Introduction

Machine learning and neural networks in particular, are currently hot topics in data processing. Many tools and platform are now easily available to work and experiment  (more...)

“What do you mean there’s line breaks in the address?” said SQLLDR

I had a large-ish CSV to load and a problem: line breaks inside some of the delimited fields.

Like these two records:

one, two, "three beans", four
five, six, "seven
beans", "eight wonderful beans"

SQL Loader simply won’t handle this, as plenty of sad forum posts attest. The file needs pre-processing and here is a little python script to do it, adapted from Jmoreland91’s solution on Stack Overflow.

import sys, csv, os
 
 (more...)

Rename multiple exported Files after using SQL Developer’s Cart to export from Oracle database

If you’re searching for “export Oracle BLOB”, the article, by Jeff Smith, titled “Exporting Multiple BLOBs with Oracle SQL Developer” using Oracle SQL Developer” is usually at the top of the search result. The SQL Developer features the Shopping Cart without using scripts to export BLOBs out of database. I don’t want to go into detail as Jeff already explained well in his post what it is and how to use it. One (more...)

Multisessioning with Python

I'll admit that I pretty constantly have at least one window either open into SQL*Plus or at the command line ready to run a deployment script through it. But there's time when it is worth taking a step beyond.

One problem with the architecture of most SQL clients is they connect to a database, send off a SQL statement and do nothing until the database responds back with an answer. That's a great model when (more...)

IoT Hackathon Part IV : Using Web Services to send Sensordata

In the previous 3 posts, building towards the eProseed IoT Hackathon, I described how to setup your Raspberry Pi, and how to use the GrovePi sensors. The used example is a small weather-station that read temperature and humidity and shows the readings on a display. That is all very nice, however, the data remains local on the Raspberry Pi so there is nothing that we can do with this information

Generating Diceware Passwords in Python

Today I’m going back to a theme from a post last year and looking at generating passwords with my favourite programming language. A tweet from Simon Brunning pointed me to Micah Lee’s article at The Intercept and my first thought was to write a function to do this in Python. So here it is;

def generate_diceware_password(word_count=6):
    import random
    word_dict = {}
    passphrase = []
    with open('diceware.wordlist.andy.txt') as f:
        for line in f. (more...)

start python learning with "Introduction to Python"

Python is programming language, that supports object-oriented, imperative and functional programming. The key is its simplicity, easy language to learn and easy moving code from development to production more quickly. It's power tool to use with Big Data. So, I believe it's a good time to learn about Python programming language.
You can find many resources about it on the Internet. I started to learn about Python by watching "Introduction to Python" By Jessica McKellar. (more...)

Generating Reasonable Passwords with Python

Thanks to a certain recent Open SSL bug there’s been a lot of attention paid to passwords in the media. I’ve been using KeePassX to manage my passwords for the last few years so it’s easy for me to find accounts that I should update. It’s also a good opportunity to use stronger passwords than ‘banana’.

My problem is that I have always resisted the generation function in KeePassX because the resulting strings are very (more...)

It’ll Come Out in the Wash

After reading this fine summary of the history of Python 3 by Nick Coghlan I was inspired to update as many of my half finished projects and miscellaneous scripts as possible. Then I looked up and I had lost several hours of my life. To save random internet strangers from the same pain as I experienced here is a catalogue of problems and how I solved them.

tl;dr – Python 3.3 on Ubuntu 12. (more...)

ipython-sql bind variables

Thanks to Mike Wilson, ipython-sql now supports bind variables!


In [12]: name = 'Countess'

In [13]: %sql select description from character where charname = :name
Out[13]: [(u'mother to Bertram',)]

Review of Learning IPython for Interactive Computing and Data Visualization

valuable but traditional

May 25, 2013 by Catherine Devlinphoto of 'Learning IPython for Interactive Computing and Data Visualization' 4 stars (of 5)

Packt Publishing recently asked if I could review their new title, Learning IPython for Interactive Computing and Data Visualization. (I got the e-book free for doing the review, (more...)

cannot import name MAXREPEAT

When I upgraded from Xubuntu 12.10 to 13.04 today, all my existing Python virtualenvs broke! Fortunately, they're just virtualenvs and easy to replace (that's kind of the point). But don't panic if you start seeing these.


$ ipython
Traceback (most recent call last):
File "/home/catherine/ve/e2/bin/ipython", line 5, in
(more...)

Speak at PyOhio

Have you responded yet to PyOhio's Call For Proposals (due date: June 1)? You should. Here's why.

Why you should speak at PyOhio

  • We need you. We have a great group of people who contribute talks, but we don't ever want that group to become a stagnant pool - we (more...)
  • ipython-sql for multi-database comparisons

    For my newest ipython-sql trick, I needed to compare some queries run across different databases. How hard would it be to get side-by-side results into tidy IPython Notebook output?

    Not hard at all, it turns out, if you're willing to violate basic principles of human decency.

    That's an itty-bitty image, (more...)

    Living in the Future

    On my morning commute today I realised that I am actually living in the future. I remember when I got involved in the PythonCard project 10 years ago one of the major questions on the mailing list was why we were building a GUI toolkit when the future was the web. It wasn’t true then but I think that it is now.

    Why do I think we have moved now? It is in large part thanks to a book I have started reading called Python for Data Analysis. I have a copy of the book in ePub format and wanted (more...)

    Separate docs for MySQL Connectors

    The MySQL documentation section has always had this Topic Guides page containing links to the docs for the various MySQL Connectors -- the official database drivers for various languages and programming technologies. That is the most convenient way to get the information for each Connector in PDF form, rather than downloading the entire Ref Man PDF. For HTML, it was more of a shortcut, because