Using ElasticSearch, Fluentd and Kibana (for log aggregation)

In my last article I describe how I installed ElasticSearch, Fluentd and Kibana (EFK).
[https://technology.amis.nl/2019/04/23/using-vagrant-and-shell-scripts-to-further-automate-setting-up-my-demo-environment-from-scratch-including-elasticsearch-fluentd-and-kibana-efk-within-minikube/]

In this article I will dive into using ElasticSearch, Fluentd and Kibana. Besides log aggregation (getting log information available at a centralized location), I will also describe how I created some visualizations within a dashboard.

Kibana 7.0 has a new sleek design, streamlined navigation, and more for an extra delightful user experience.
[https://www.elastic.co/products/kibana]

Elasticsearch

Elasticsearch (more...)

The Pipelined Pancake Party – demonstrating the Power of Parallelization and Pipelining in JavaScript with Promises and Asynchronous Generators

This article is an attempt to demonstrate the performance gains – and programming elegance – that is at our disposal with the advent of asynchronous generators in ES 2018 (JavaScript), for example in Node 10 and later. With asynchronous generators and Promises, we can implement parallel, asynchronous and pipelined processing that allow us to process a result set ahead of time.

The business case at hand… a pancake party (this may be a very Dutch (more...)

JavaScript Pipelining using Asynchronous Generators to implement Running Aggregates

image

As of ES 2018 (recent browsers or Node 10), JavaScript support asynchronous generators. Generators are functions that return a set of values, one value at a time. These values can be processed inside the code that invokes the generator immediately, as soon as they become available. There is no need to wait for the entire result set to be composed first. In cases where the result set is huge or even never ending, this is (more...)

Jupyter Notebook for retrieving JSON data from REST APIs

If data is available from REST APIs, Jupyter Notebooks are a fine vehicle for retrieving that data and storing it in a meaningful, processable format. This article introduces an example of a such a dataset:

image

Oracle OpenWorld 2018 was a conference that took place in October 2018 in San Francisco. Over 30,000 attendees participated and visited some 2000 sessions. Raw data from the session catalog is available from an API – a REST service that (more...)

The Full Oracle OpenWorld and CodeOne 2018 Conference Session Catalog as JSON data set (for data science purposes)


Oracle OpenWorld and CodeOne 2018 are two co-located conferences that took place in October 2018. Some 2000 sessions presented by over 2500 presenters form the core of these conferences. Many details are known about each of the sessions and the speakers – from title, abstract, room (size), date and time, session slides, session type and key topics to first name, last name, Twitter handle, picture, company, job title and bio. image

A data set is available (more...)

JavaScript Generators – How functions return a stream of results, yielding one result at a time

It was through inspecting some Python code that relied quite heavily that I suddenly realized the beauty of the ES6 concept of generators and the yield keyword. A generator function does not return its result all at once but instead an iterator that can be read from, one value at a time. A for loop can be constructed that iterates over the results from a function – but the function does not have to create (more...)

Test Remote Asynchronous Request Response services

A few years ago, I described how you can test Asynchronous Request Response services.

The thing with Asynchronous Request Response services is, as I used to describe it, that they're in essence two complementary Request-Only (Fire and Forget) services. That is, the client submits a request to the Asynchronous Request Response service, and at a certain point waits for the response by listening to an endpoint.

To make this work, the responding Asynchronous Request Response (more...)

Using Vagrant and shell scripts to further automate setting up my demo environment from scratch, including ElasticSearch, Fluentd and Kibana (EFK) within Minikube

For training and demo purposes, on my windows laptop, I needed an environment with a guest Operating System, Docker and Minikube available within an Oracle VirtualBox appliance.

So, in a series of articles up till now, I described the following:

Newly released: Oracle Exadata X8-2 – bigger disks for saving money, expanding capacity

imageEarlier this month, Oracle announced the availability of the latest Exadata machine, the X8-2. As well as the option to swap in big 14 TB disks in older versions of Exadata that were shipped with far smaller disks, allowing these older systems to substantially increase there capacity.

Our AMIS resident Exadata expert Michael Huisman analyzed the key aspects of Oracle’s announcement.

The changes in specifications compared to X7 are not huge. The most eye catching (more...)

Creating a Report of available ASM candidate disks from the OEM Repository

A while ago I was reviewing the metric extensions of our Enterprise Manager Cloud Control 13.2.0.0, we use to administer a couple of hundred databases. All acceptance and production databases are installed on Linux RAC-clusters with underlying ASM storage. One of the custom reports summarized the size and number of LUN’s, using a query I had found on the internet:

SELECT target_name,

diskgroup,

MAX (DECODE (seq, 7, VALUE))               as REDUNDANCY,

MAX (DECODE (more...)

Split your Vagrant provisioners

For a while now, I'm quite into Vagrant in combination with VirtualBox. A few years ago I started with trying to script FMW environments, and since my discovery, and resulting fancy, of Vagrant, I also created a project for creating and provisioning a SOA Suite box.

Until now, my project all had one shell-type provisioner looking like:
  config.vm.provision "shell", inline: <<-SHELL
export SCRIPT_HOME=/vagrant/scripts
. $SCRIPT_HOME/install_env.sh
echo _______________________________________________________________________________
echo 0. Prepare Oracle Linux
$SCRIPT_HOME/0. (more...)

Extremely convenient way to run free Oracle Database 18c on your laptop – or anywhere else – using Vagrant & Virtual Box

imageOracle Database 18c XE (eXpress Edition) is the free community edition of the enterprise edition of Oracle Database. Oracle Database XE has almost all features of Oracle Database; however, it is limited in non-functional terms (2 CPUs, 2 GB memory, 12 GB data volume). Oracle is generous in the terms under which XE can be used:

“We grant you a nonexclusive, nontransferable limited license to use the programs for: (a) purposes of developing, prototyping and (more...)

INS-08106 using 12.1 dbua on a server with 12.2 databases

When you want to upgrade a 11.2 database to 12.1 on a (linux) server with 12.2 databases, you get a INS-08106 error when using the 12.1 dbua when loading the select database Page:


image

Reason:

The 12.1 dbua does not know version 12.2

Solution:

  1. Use DBUA in silent mode
  2. Comment out the 12.2 databases in /etc/oratab: #DATABASE:/app/oracle/product/12.2.0.1/dbhome_1:N

The post INS-08106 using 12.1 dbua on a (more...)

ADF Performance Monitor: Error Diagnostics

Application errors are often hard to retrieve, or take a lot of time to resolve. When you are suffering from errors, and have a lack of clarity when errors happen, you would like to have useful error diagnostics for analysis.

The ADF Performance Monitor automatically captures detailed diagnostics for each and every error/exception occurrence. You can view your errors to see the highest priority issues your team should focus on. This blog shows the renewed (more...)

My Seemless Linux Desktop using VirtualBox, Vagrant, MobaXterm and TotalCommander

Years ago I played around with VMWare Unity Mode in VMWare player or VirtualBox's Seemless mode. In those modes you start an application on your virtual machine, but the windows appear as running on your host system. Back in the days I ran OpenSuse on my laptop, and had Windows XP or 7 on a Virtual Machine for those cases I had to run PowerPoint.

I wasn't too enthousiastic about those modes. Of the two (more...)

Updating running Kubernetes resources from the kubectl commandline using the patch command

All too often I found myself editing Kubernetes resource definitions in the Kubernetes Dashboard. Modifying for example the type of a service. Typically I would git clone yaml files, run  them in a single statement and then manually change a few details.

image

After just the tiniest of search efforts, I ran into the kubectl patch command, that is created for just this purpose.

kubectl patch svc eventbuspublisherservice -p ‘{“spec”: {“type”: “NodePort”}}’ –namespace my_space

This will (more...)

Running Apache Kafka on Minikube

imageI frequently want to demonstrate the working of Apache Kafka and/or provide colleagues and other audiences with an environment for a workshop that involves Apache Kafka. I have been struggling to find  a good way to get Kafka to function properly on Minikube – the standalone, single node Kubernetes cluster for experiments. This weekend, I have worked with a set of K8S resource files that gives me what I need. The GitHub repository created by (more...)

6 tips to make your life with Vagrant even better!

HashiCorp Vagrant is a great tool to quickly get up and running with a development environment. In this blog post I’ll give some tips to make your life with Vagrant even better! You can find an example which uses these tips here.

Tip 1: The disksize plugin

Ever created a VM with a specific size for the disk and after using it for a while, you find out you would have liked to have a (more...)

SOASuite 12c upgrade – Composite DVMs

Today I found something curious in a composite upgraded from 11g to 12c, regarding DVMs. I sometimes use DVMs in BPEL to prevent the use of complex xpath expressions with many conditions. For instance, if I need to know if a JMSType is in a certain range and if it is I need to continue, I can create a DVM that has those JVMTypes correlated to an indicator.

Now, in 12c we have a new project (more...)

JavaDB not bundled anymore with JDK 8, as of U181

Today I was struggling with helping a colleague with a deployment of a SOA Project of his.
I couldn't get it deployed. It seemed I hit the problem described here. However when trying to connect to my Derby DB I got the following error:

I was very surprised. I checked and double checked my config. And check the library:
So, I checked those folders and found that they're not existing!
Now searching around I found (more...)