Database Technologies

Its being two months, that I didn't blog on any topic. So, I thought to share some information on PEM. This post is for users, who always look for type of inbuild alerts of PEM. In this Blog, I am listing categories of Alerts and List of alerts in each category, which one can find in Postgres Enterprise Manager 2.1. Also, if you have list of alerts then you can decide which alert you would like to configure. Postgres Enterprise Manager (PEM) is very useful tool for monitoring PostgreSQL and it has all the alerts which is require for monitoring postgresql. User can define this alerts in following categories: 1. Server Level Alerts. 2. PG Cluster Level Alerts. 3. Database Level Alerts. 4. Schema Level Alerts 5. Table Level Alerts. All the alerts which has been made/defined keeping in mind of postgreSQL. Let see what are the alerts in each categories: 1. Server Level Alerts: These alerts are made for monitoring Server Components like Memory,Disk and CPU and Server Level moni

This has been asked me many times. Many users has asked me how they can create READ-ONLY user, READ-ONLY Database or How they can create backup user which can be use only for backup (no modification) Well answer is in parameter called default_transaction_read_only. If you want to make a user READ-ONLY, then you can follow steps given below: 1. CREATE normal user. 2. Use ALTER USER command to set this parameter for this user as given below: ALTER USER set default_transaction_read_only = on; 3. GRANT SELECT to this user for tables/object using GRANT. And you are good to go. Below is snapshot: postgres=# create user readonly password 'test'; CREATE ROLE postgres=# alter user readonly set default_transaction_read_only = on; ALTER ROLE postgres=# GRANT select on employees to readonly; GRANT edbs-MacBook-Pro:data postgres$ psql -U readonly -W Password for user readonly: psql (9.1.1) Type "help" for help. postgres=> select * from employees ; employee_name | e

1. True serializable isolation level:  Before version 9.1, PostgreSQL did not support a full serializable isolation level. A request for serializable transaction isolation actually provided snapshot isolation. This has well known anomalies which can allow inconsistent views of the data during concurrent transactions; although these anomalies only occur when certain patterns of read-write dependencies exist within a set of concurrent transactions. Where these patterns exist, the anomalies can be prevented by introducing conflicts through explicitly programmed locks or otherwise unnecessary writes to the database. Snapshot isolation is popular because performance is better than serializable isolation and the integrity guarantees which it does provide allow anomalies to be avoided or managed with reasonable effort in many environments.  Following is a link on more details: http://wiki.postgresql.org/wiki/SSI 2. INSERT/UPDATE/DELETE in WITH CLAUSE.  Now in 9.1, User would be able i

Since, PostgreSQL 9.1 has already been released and there are lot of new features added in it, so, I thought to blog about each features. Today, I am discussing about Synchronous and Asynchronous Replication supported in PostgreSQL 9.1. For Asynchronous Replication, user can use following method: 1. Change Following Parameters in postgresql.conf file of Primary Database: archive_command = cp -i %p /Users/postgres/archive/%f archive_mode = on max_wal_senders = 3 # Maximum 'wal_senders', processes responsible for managing a connection with a standby server wal_keep_segments = # How many WAL segments (=files) should be kept on the primary, whatever may happen (you won't have to copy them manually on the standby if the standby gets too far behind) wal_level = hot_standby 2. Make Following changes in pg_hba.conf host replication postgres [Ipv4 address of Standby Server]/32 trust host replication postgres [Ipv4 address of Master Server]/32 tr

Today, I thought to share my Training slides on FTS (Full Text Search) in PostgreSQL 9.0. I have used this presentation on giving training. Enjoy :). Full Text Search Training

Depsez has Started a New project called Curo, which is currently made for Unix/Linux and is in initial phase. So, I thought to look at how it works and how normal user, who knows using psql, can benefit from it. Before Installing Curo, user has to make sure, he has postgreSQL client: psql and following package installed on his Linux/Unix System: 1. Bash 2. dialog If you have above required packages on your machine, then you can move for Curo installation. Curo Installation is very simple. Following are the steps which can be use to install Curo on Unix/Linux System: 1. Download the Curo from Following link: https://github.com/omniti-labs/curo Above link will give you file similar to given below: omniti-labs-curo-48a4f31.tar.gz 2. Untar the file using following command: tar -xvf omniti-labs-curo-48a4f31.tar.gz 3. Copy omniti-labs-curo-48a4f31 to users home directory as given below: cp omniti-labs-curo-48a4f31 ~/.curo 4. Now configure .inputrc and .editrc file as given

I had seen people ask Questions about finding the edb-jdbc/postgresql driver version with Database Version. So, I thought to give one java script which can be use to find Database Details with edb-jdbc version. Following is a JAVA Code which can be use to find the Database Version and EDB-JDBC/postgresql-jdbc Version: File Name: DBinfo.java import java.sql.*; public class DBinfo { public static void main(String[] args) { try { Class.forName("com.edb.Driver"); Connection con = DriverManager.getConnection("jdbc:edb://localhost:5444/edb", "enterprisedb","edb"); // Advanced Server Database Connection Information DatabaseMetaData dbmd = con.getMetaData(); System.out.println("===== Database info ====="); System.out.println("DatabaseProductName: " + dbmd.getDatabaseProductName() ); System.out.println("DatabaseProductVersion: " + dbmd.getDatabaseProduc

Logrotator is a small C program which is well written and available on most of the UNIX based OS. It allows automatic rotation, compression of logfiles and removal of old logfiles. This small file can be use to manage applications logfiles. It can also use for managing slony logfiles. For managing logfiles using logrotator user has to make a logrotator conf file, and can be scheduled in crontab for managing logile. Let see how you can make a logrotator configuration file for slony. I have slony master/slave daemon running on my slave maching has following location for slony logfiles: /opt/PostgresPlus/9.0AS/slony_test/slony1/node1 -- For master /opt/PostgresPlus/9.0AS/slony_test/slony1/node2 -- For slave. For managing logfile, user can make logrotator configuration file similar to given below: Configuration filename: /etc/logrotate.d/slonylogfiles "/opt/PostgresPlus/9.0AS/slony_test/slony1/node[12]/*.log" { rotate 5 mail vibhor.aim@gmail.com

Today, I though to mention some of Oracle workarounds which can be use for Migration of Oracle Database to PPAS 9.0(EnterpriseDB). 1. UTL_RAW.CAST_TO_RAW This is a Oracle Packaged Function, which can be use to convert VARCHAR2 value into RAW Value. Lets see its example in Oracle: SQL> select utl_raw.cast_to_raw('TESTING TESTING') from dual; UTL_RAW.CAST_TO_RAW('TESTINGTESTING') -------------------------------------------------------------------------------- 54455354494E472054455354494E47 Since PPAS 9.0, doesn't have this package in 9.0, therefore user can use postgresql function decode() for converting of varchar datatype to RAW/bytea data. Following is an example: edb=# select decode('TESTING TESTING','escape') from dual; decode ---------------------------------- \x54455354494e472054455354494e47 (1 row) which is a same output. 2. UTL_RAW.CAST_TO_VARCHAR2 Using this packaged function user can conver

This has been asked many times, so I thought to write steps for enabling ssl: Following are steps, which can be use to enable ssl in postgreSQL: 1. Generate a passphrase protected certificate using following command: openssl req -new -text -out cert.req Snapshot is given below: Generating a 1024 bit RSA private key ....................++++++ ...................................................++++++ writing new private key to 'privkey.pem' Enter PEM pass phrase: Verifying - Enter PEM pass phrase: ----- You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN. There are quite a few fields but you can leave some blank For some fields there will be a default value, If you enter '.', the field will be left blank. ----- Country Name (2 letter code) [AU]: State or Province Name (full name) [Some-State]: Locality Name (eg, city) []: Organization Name

One of my Colleague had asked how he can use Nagios for PostgreSQL Monitoring. So, I thought of writing steps to show the blog readers about nagios Configuration for PostgreSQL/PPAS. Following are the steps: 1. Download Nagios Core wget http://nagios.sourceforge.net/download/cvs/nagios-HEAD.tar.gz 2. For Installing Nagios, following packages would be required,so make sure you have them installed on your server. gcc make binutils cpp libpq-dev libmysqlclient15-dev libssl0.9.8 libssl-dev pkg-config apache2 libgd2-xpm libgd2-xpm-dev libgd-tools libpng12-dev libjpeg62-dev perl libperl-dev libperl5.8 libnet-snmp-perl 3. Create nagios user and group as given below: [root@localhost]# useradd nagios [root@localhost]# passwd nagios [root@localhost]# groupadd nagcmd [root@localhost]# usermod -G nagcmd nagios [root@localhost]# usermod -G nagcmd apache 4. Now, untar Nagios Core/Nagios Plugin using following command: tar -xvf nagios-HEAD.tar.gz 5. Now configure Nagios

I have seen question, like "How to make Database Link from PostgreSQL to Oracle?", always floats in PostgreSQL Community Forum. So, I thought to do some research on it and write a Blog. Cybertec (One of The PostgreSQL Database Company) has released a PostgreSQL Module ODBC Link, using which user can make Database link to any other database, including Oracle, MS SQL Server, PostgreSQL etc, which have ODBC compliant Data source. Lets see how you can make ODBC Connection from PostgreSQL to any ODBC compliant data source and fetch data. Installation of this module is very simple. Following are the steps which user can follow: 1. Install the unixODBC driver on your linux machine. user can use following useful link for Downloading and Installing the unixODBC Driver: http://www.unixodbc.org/ 2. Download ODBC-Link from following location: http://www.cybertec.at/download/odbc_link/ODBC-Link-1.0.4.tar.gz 3. Untar the downloaded file as given below: tar -zxvf ODBC-Link-1.0.4.

Following are some of the important Process of PostgreSQL/Postgres Plus Advanced Server. vibhore@ubuntu:~/edb/edb-postgres$ ps -eaf|grep postgre 1002 1080 1 0 06:07 ? 00:00:00 /opt/PostgresPlus/8.4AS/bin/edb-postgres -D /opt/PostgresPlus/8.4AS/data (Mandatory) 1002 1101 1080 0 06:07 ? 00:00:00 postgres: logger process (Mandatory) 1002 1103 1080 0 06:07 ? 00:00:01 postgres: writer process (Mandatory) 1002 1104 1080 0 06:07 ? 00:00:01 postgres: wal writer process (Mandatory) 1002 1105 1080 0 06:07 ? 00:00:00 postgres: autovacuum launcher process (Optional if autovacuum is on) 1002 1106 1080 0 06:07 ? 00:00:00 postgres: stats collector process (Mandatory) Above is an snapshot of the processes which exist in PostgreSQL/Postgres

Generally DBA, has to login in system to monitor the activities by querying the pg_stat_activity veiw. However if someone wants to do continous monitor for every 2 seconds. He has to run SQL against pg_stat_activity every 2 seconds. So, For such activity, I have made following shell script which can be use with watch command on linux to monitor the activities. #!/bin/bash ## Prepared by Vibhor Kumar (EDB). Q="select procpid,now() - query_start as long, waiting, current_query from pg_stat_activity where procpid <> pg_backend_pid() limit 10;" while getopts "b:U:d:p:" opt; do case $opt in b) BIN=$OPTARG;; U) PGUSER=$OPTARG;; d) DB=$OPTARG;; p) port1=$OPTARG;; *) echo "Usage: $0 -b -d -U -mp -sp ";; esac done if [ -z $BIN ];then echo "Usage: $0 -b bin directory -d database name -U user -p port" exit fi if [ -z $PGUSER ];then echo "Usage: $0 -b bin directory -d database name -U user -p por

This is a Question which I had seen people generally ask, when they Move their DBs to PG or PGPlus Advanced Server and Look for Workaround of Oracle Feature of Traverse of Directed Graph in PG. Oracle Provides CONNECT BY NOCYCLE for traversing a Directed Graph. As mentioned in Following link: http://www.adp-gmbh.ch/ora/sql/connect_by_nocycle.html However, CONNECT BY NOCYCLE implementation, some times miss some paths for Traversing Graph as you can see in following Example: create table directed_graph ( node_from char(1), node_to char(1) ); insert into directed_graph values ('A', 'C'); insert into directed_graph values ('A', 'B'); insert into directed_graph values ('B', 'E'); insert into directed_graph values ('C', 'H'); insert into directed_graph values ('H', 'I'); insert into directed_graph values ('I', 'D'); insert into directed_graph values ('D', 'F'); insert

Lot of People think that COPY is only a Server Based Command. However its not completely true. User can copy the data of a table at client side using COPY Command. The Difference is only in syntax usage. User has to use backSlash before COPY Command as given below: testdb=# select * from test; id ---- 1 2 3 (3 rows) testdb=# \copy test to '/tmp/test.copy' testdb=# \q We can see output of a file at Client side as given below: cat /tmp/test.copy 1 2 3 Similarly User can also copy the data in a file to Server using \COPY as given below: \COPY test from '/tmp/test.copy' iclive1460=# \COPY test from '/tmp/test.copy' iclive1460=# select * from test; id ---- 1 2 3 1 2 3 (6 rows) Note:: One should remember while using \COPY Command, one should not terminate command with semicolon ';'

SQL Injection is method of Hacking Data inside the Database. Using SQL Injection, Hackers do multiple Attempt of SQLs to understand the Structure of tables and Try to get the hint of data inside a table. For example, if a Hacker knows that UserName and Password of a Website are stored in a database table say member, then he can write query something like given below to fetch all the data inside the function as given below: select * from tablename where 1=1; Or select * from tablename where 'x'='x'; Similarly, suppose Hacker wants to know the structure (Columns) of any table, then he can try multiple attempts of finding the column name by using the HAVING OR GROUP BY CLAUSE and can find the Columns Names Based on Error. As given below: postgres=# select * from test_sql_injection group by 1 having 1=1; ERROR: column "test_sql_injection.col" must appear in the GROUP BY clause or be used in an aggregate function LINE 1: select * from test_sql_injection gro