Posts

Showing posts from 2018

sql maven plugin write file in target encoding.

Sql-maven-plugin is used to execute database queries as part of build steps. It can also generate files from database queries. It is an excellent tool when it comes to generating file where certain application parameters are saved in a database and are needed by an application as properties. Some applications save configuration parameters in the database but do not want to read it from the database at runtime, because of various business or architecture compliances. These applications rather produce configuration files from the database as part of the build step and use them as application properties as resource bundle. One good use is for locale messages. applications supporting multiple languages, generate message files of different locale from database tables, which are usually managed by business teams. Locale support needs specific encodings, which is not supported by out of the box by popular sql-maven-plugin from codehaus. The maven plugin code at GitHub.com has a pull

uscis case dump in a given range

I am providing a simple Python script, that one can use to download USCIS cases in agiven range. The case data may provide you an idea on time of action in your case. Do not abuse the script and put undue pressure on USCIS system. Your IP will get blocked for abuse for long ranges. An abuse may be seen as DOS (denial of service) attack and may invoke felony proceedigs. To use the code, edit  "START_CASE", "END_CASE" and "fname" vaiables. Read comments for details. # Use this code very responsibly and do not abuse the system. Abusing the system may result in a felony. import requests #pip install htmldom from htmldom import htmldom import re import os CASE_TYPE = "I-485" # put the start case number in full as in the example below START_CASE = 'MSC2290530000' #MSC2290532863 # put the end case number in full as in the example below END_CASE = ' MSC2290540000 ' # file location to save the results fname = "case-with-date.c

pandas dataframe add missing date from range in a multi-dimensional structure with duplicate index

This solution demonstrates how to fill in the missing dates in a given range in a multi-index pandas dataframe. The complexity is added by the presence of duplicate dates in the given data, where the date is considered as an index. In case you try to reindex a data frame with duplicate indexes, you will get the following error. ValueError : cannot reindex from a duplicate axis To resolve this situation and to achieve the end goal of refitting dataset with missing indexes, following pseudo code can be used. Read multidimensional data into pandas dataframe (dataset), with date column as an index (only one index). Transform dataframe index created above into datetime index type Create a new dataframe (d) with the required date range, and value of other records as null Append 'd' into 'dataset' Set index of 'dataset' to include more column to create a multi-level index Reindex to 'dataset', and fill the desired value. The example python co

Python - cannot install mysqlclient with pip

I was trying to intsall mysqlclinet to use following python module oin my program   import MySQLdb as db   As usual it will not work as "mysqlclinet" is not installed. So I tried to install it using pip.   sudo pip install mysqlclient   But it failed with following error. building '_mysql' extension i686-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -Dversion_info=(1,2,4,'beta',4) -D__version__=1.2.4b4 -I/usr/include/mysql -I/usr/include/python2.7 -c _mysql.c -o build/temp.linux-i686-2.7/_mysql.o -DBIG_JOINS=1 -fno-strict-aliasing -g -DNDEBUG _mysql.c:29:20: fatal error: Python.h: No such file or directory compilation terminated. error: command 'i686-linux-gnu-gcc' failed with exit status 1   I was using python version 3.7.0. I tried everything mentioned in various posts online, but nothing worked.   Many posts suggested using follwoing command:   sudo apt-get install python3-d

Private Cloud- An insight

Image
Introduction Cloud concept is not a new phenomenon. Since the beginning of client-server model, the cloud was there in in its evolutionary phase. In 1964 on OS/360 where the request was to run a job, and the response was the output, it was similar to SaaS (software as service). In late 1994 when Netscape browser was launched (a.k.a Mosaic) concept of SaaS picked the pace and transformed the internet as never seen before. Soon we were all checking our emails, browsing information over the internet on remote servers. Many frameworks were created to provide the content in best possible manner. This led to the growth of multi-threaded request processing capabilities on remote servers (APACHE web server become a default part of Linux distributions). Application development picked the pace and new frameworks, solutions were adapted to generate information in a dynamic way (JAVA multi-threading enabled web servers to serve multiple requests parallel from a single instance of the server