Creating a Pivotal GemFireXD Data Source Connection from IntelliJ IDEA 13.x

In order to create a Pivotal GemFireXD Data Source Connection from IntelliJ 13.x , follow the steps below. You will need to define a GemFireXD driver , prior to creating the Data Source itself.

1. Bring up the Databases panel.

2. Define a GemFireXD Driver as follows


3. Once defined select it by using the following options. Your using the Driver you created at #2 above

+ -> Data Source -> com.pivotal.gemfirexd.jdbc. (more...)

Variable in list with Postgres JDBC and Greenplum

I previously blogged on how to create a variable JDBC IN list with Oracle. Here is how you would do it with Pivotal Greenplum. Much easier , without having to write a function. In the Greenplum demo below we use the any function combined with string_to_array

http://theblasfrompas.blogspot.com.au/2008/02/variable-in-list-with-oracle-jdbc-and.html

Code as follows
  
import java.sql.*;
import java.sql.DriverManager;

/**
* Created by papicella on 4/09/2014.
*/
public class VariableInListGreenplum
{

public (more...)

REST with Pivotal GemFire 8.0

Pivotal GemFire 8.0 now includes REST support. You can read more about it as follows

http://gemfire.docs.pivotal.io/latest/userguide/gemfire_rest/book_intro.html#concept_7628F498DB534A2D8A99748F5DA5DC94

Here is how we set it up and some quick examples showing how it works with some Region data
In the example below I have PDX setup for the cache servers as shown below.
  
<!DOCTYPE cache PUBLIC
"-//GemStone Systems, Inc.//GemFire Declarative Caching 8.0//EN"
"http://www.gemstone.com/dtd/cache8_0.dtd">
<cache>
<pdx read-serialized="true">
<pdx-serializer>
(more...)

Dept/Emp POJO’s with sample data for Pivotal GemFire

I constantly blog about using DEPARTMENT/EMPLOYEE POJO'S with sample data. Here is how to create a file with data to load into GemFire to give you that sample set.

Note: You would need to create POJO'S for Department/Empployee objects that have getter/setter for the attributes mentioned below.

Dept Data

put --key=10 --value=('deptno':10,'name':'ACCOUNTING') --value-class=pivotal.au.se.deptemp.beans.Department --region=departments;
put --key=20 --value=('deptno':20,'name':'RESEARCH') --value-class=pivotal.au.se.deptemp.beans.Department --region=departments;
put --key=30 --value=('deptno':30,'name':'SALES') --value-class=pivotal.au.se. (more...)

Using HAWQ with PHD service in PCF 1.2

The following demo shows how to use the PCF 1.2 PHD service with HAWQ by loading data into the PCF PaaS platform.

1. First lets setup our ENV to use the correct version of HADOOP on our local laptop.

export HADOOP_INSTALL=/Users/papicella/vmware/software/hadoop/hadoop-2.0.5-alpha
export JAVA_HOME=/System/Library/Frameworks/JavaVM.framework/Versions/CurrentJDK/Home

export PATH=$PATH:$HADOOP_INSTALL/bin:$HADOOP_INSTALL/sbin
export HADOOP_OPTS="$HADOOP_OPTS -Djava.net.preferIPv4Stack=true"

export HADOOP_OPTS="$HADOOP_OPTS  -Djava.awt.headless=true -Djava.security.krb5.realm=-Djava.security.krb5.kdc="

export YARN_OPTS="$YARN_OPTS -Djava.security.krb5.realm=OX.AC.UK -Djava. (more...)

Pivotal Cloud Foundry Installed lets create an ORG / USER to get started

I installed Pivotal Cloud Foundry 1.2 recently and the commands below is what I run using the CLI to quickly create an ORG and a USER to get started with. Below assumes your connected as the ADMIN user to set a new ORG up.

Cloud Foundry CLI Commands as follows

cf api {cloud end point}
cf create-org pivotal
cf create-user pas pas
cf set-org-role pas pivotal OrgManager
cf target -o pivotal
cf create-space development
(more...)

Pivotal GemFireXD*Web, Web based Interface For GemFireXD

Pivotal GemFire XD bridges GemFire’s proven in-memory intelligence and integrates it with Pivotal HD 2.0 and HAWQ. This enables businesses to make prescriptive decisions in real-time, such as stock trading, fraud detection, intelligence for energy companies, or routing for the telecom industries.

You can read more about how GemFireXD and it's integration with PHD here.

https://www.gopivotal.com/big-data/pivotal-hd

While development team worked on GemFireXD I produced another open source web based tool named GemFireXD*Web. (more...)

Creating some Pivotal Cloud Foundry (PCF) PHD services

After installing PHD add on for Pivotal Cloud Foundry 1.1 I quickly created some development services for PHD using the CLI as shown below.

[Tue Apr 15 22:40:08 papicella@:~/vmware/pivotal/products/cloud-foundry ] $ cf create-service p-hd-hawq-cf free dev-hawq
Creating service dev-hawq in org pivotal / space development as pas...
OK
[Tue Apr 15 22:42:31 papicella@:~/vmware/pivotal/products/cloud-foundry ] $ cf create-service p-hd-hbase-cf free dev-hbase
Creating service dev-hbase in org pivotal / space development as pas...
OK
[Tue Apr (more...)

Pivotal Greenplum GPLOAD with multiple CSV files

I recently needed to setup a cron script which loaded CSV files from a directory into Greenplum every 2 minutes. Once loaded the files are moved onto Hadoop for archive purposes. The config below shows how to use GPLOAD data load utility which utilises GPFDIST.

1. Create a load table. In this example the data is then moved to the FACT table once the load is complete
  
drop table rtiadmin.rtitrans_etl4;

CREATE TABLE rtiadmin.rtitrans_etl4 (more...)

Pivotal Cloud Foundry using App Direct "newrelic" Monitoring Service

PCF AWS marketplace provides app direct services and in this example I am going to use the "newrelic" monitoring service to monitor my spring based java application. It's really this simple.

1. Create a service as shown below.

[Tue Mar 04 17:19:34 papicella@:~/cfapps/spring-travel ] $ cf create-service newrelic standard dev-newrelic

2. Create a manifest.yml for my spring application which uses the new relic service above.

applications:
- name: pas-springtravel 
  memory: 1024M 
(more...)

Deploying Spring MVC application to Cloud Foundry from IntelliJ IDEA

I previously showed how to create a connection in IntelliJ IDEA to Cloud Foundry v2 in the post below.

http://theblasfrompas.blogspot.com.au/2014/02/intellij-idea-version-13-now-includes.html

With a Cloud Foundry CLOUD connection we can now PUSH our application directly from the IDE as shown below.

1. Create a run configuration for your project as shown below. We also specify the memory and number of instances on this page as part of the push  / deployment process.



2. (more...)

IntelliJ IDEA version 13 now includes CloudFoundry Connection

Just installed IntelliJ IDEA version 13 and found that it now includes a CloudFoundry connection type. You define it under IDE settings as shown below.


Will test deploying to the publicly hosted AWS Cloud Foundry using this connection at some stage.

PCF (Pivotal Cloud Foundry) cf push multiple applications using manifest file

By creating a manifest as follows we can push multiple applications in one go as shown below.

1. manifest.yml

applications:
- name: pas-props
  memory: 256M
  instances: 1
  host: pas-props
  domain: cfapps.io
  path: ./props.war
- name: pas-httpsession
  memory: 256M
  instances: 1
  host: pas-httpsession
  domain: cfapps.io
  path: ./haclusterdemo.war

2. Push as follows

[Thu Feb 20 14:48:45 papicella@:~/vmware/pivotal/products/cloud-foundry/apps/other ] $ cf push -f (more...)

PCF (Pivotal Cloud Foundry) Using the new CLI to bind services from a cf push

I previously blogged how to PUSH an application into PCF , then bind services as shown below.


If you want to do this in one step simply create a manifest file as shown below. This example isn't the spring books application but shows you what your manifest.yml would look like with services included so you could then create one similar to that below.

1. Create a manifest.yml file (more...)

PCF (Pivotal Cloud Foundry) new command line client

I decided to install the new CF v6 CLI for PCF (Pivotal Cloud Foundry). Given this will be the new CLI for PCF here is how it differs to the old Ruby CLI. In this example I deploy / push a spring based application and bind it to existing services.

To download and install the new CLI v6 use the link below.

http://docs.cloudfoundry.com/docs/using/managing-apps/cf/go-cli.html

In this example we have the following 2 services (more...)

Pivotal Cloud Foundry (PCF) binding a User Defined service for Oracle 11g r2

In this example below I show how you would create a user defined service for Oracle 11g R2 which is running outside of the PCF instance and how your applications would bind to that service. The example below is from the spring-music sample application which is a spring based web application.

https://github.com/cloudfoundry-samples/spring-music

1. First lets create our user defined service for Oracle , here we just use a simple URI which the Spring Cloud (more...)

Spring MVC with Thymeleaf HTML template pages

Thymeleaf offers a set of Spring integrations that allow you to use it as a full-featured substitute for JSP in Spring MVC applications.

So here is a simple example just so we can compare how things differ in HTML versus JSP, does remind me of JSF / Viewlets to be honest.

1. Let start with what our simple project looks like in IntelliJ


2. The web.xml is defined as follows
  
<web-app version="2.4"
xmlns="http://java. (more...)

SpringXD : Pre-Packaged Batch Jobs Import CSV Files to GemFireXD

Spring XD comes with several batch import and export modules. You can run them out of the box and show using the "Import CSV Files to JDBC (filejdbc)" as shown below.

This example show's inserting data into GemFireXD, you can use any RDBMS which supports a JDBC driver.

Note: XD_BASE = /Users/papicella/vmware/software/spring/spring-xd-1.0.0.M4

1. Configure connection to RDBMS in "$XD_BASE/xd/config/batch-jdbc-import.properties".

# Setting for the JDBC batch import job module
driverClass=com. (more...)

SpringXD FILE/TAIL input ingestion / JDBC Sink output to GemFireXD

The following examples shows what we can do out of the box with SpringXD TAIL input source and JDBC sink as the output source. This can all be done without writing and code.

Assumes you have SpringXD installed and GemFireXD installed. See the getting stared guide which explains how to setup SpringXD. As for a database output for JDBC you can use any RDBMS here so you could use MYSQL , ORACLE etc if you (more...)

Pivotal GemFireXD provides a graphical dashboard for monitoring known as Pulse

Like GemFire 7 , GemFireXD now includes Pulse. GemFire XD Pulse is a Web Application that provides a graphical dashboard for monitoring vital, real-time health and performance of GemFire XD clusters, members, and tables.

Use Pulse to examine total memory, CPU, and disk space used by members, uptime (more...)