So you’ve dumped 150 GiB of JVM heap, now what?

150 GiB worth of JVM heap dump is laying on hard drive and I need analyze specific problem detected in that process.

This is a dump of proprietary hybrid of in-memory RDBMS and CEP system, I'm responsible for. All data are stored in Java heap, so heap size of some installation is huge (400 GiB heap is largest to the date).

Problem of analyzing huge heap dumps were on my radar for some time, so (more...)

Custom Password Policy Validation in OpenIDM

My customer needed to ensure that passwords contained at least one ‘special character’ when a new password was created in OpenIDM.

I borrowed heavily from the provided samples but had to figure out the correct regexp.

Here is the function that I used to implement this:

function atLeastXSpecialChars(fullObject, value, params, property) {

isRequired = _.find(this.failedPolicyRequirements, function (fpr) {
return fpr.policyRequirement === “REQUIRED”;
}),
isNonEmptyString = (typeof(value) === “string” && value.length),
valuePassesRegexp (more...)

Binary search – is it still most optimal?

If you have a sorted collection of elements, how would you find index of specific value?
"Binary search" is likely to be your answer.
Algorithms theory is teaching us what binary search is most optimal algorithm for this task with log(N) complexity.
Well, hash table can do better, if you need to find key by exact match. In many cases, though, you have reasons to have your collection sorted, not hashed.

On my job, I'm (more...)

Introduction to MongoDB Security

View it on my new blog Last week at the Paris MUG, I had a quick chat about security and MongoDB, and I have decided to create this post that explains how to configure out of the box security available in MongoDB. You can find all information about MongoDB Security in following documentation chapter: http://docs.mongodb.org/manual/security/ In this post, I won't go into the detail about

Moving My Beers From Couchbase to MongoDB

See it on my new blog : here Few days ago I have posted a joke on Twitter Moving my Java from Couchbase to MongoDB pic.twitter.com/Wnn3pXfMGi — Tugdual Grall (@tgrall) January 26, 2015 So I decided to move it from a simple picture to a real project. Let’s look at the two phases of this so called project: Moving the data from Couchbase to MongoDB Updating the application code to use

The Year 2014 in Review

Once each year completes I try to do a run down of which posts have garnered the most attention during the last 12 months, both on this blog and on the Fusion Weekly newsletter. Here are the most viewed stories from 2014.

Fusion Weekly Newsletter

Our Fusion newsletter is more than a year newer than our PeopleSoft one and consequently the subscriber base is a lot smaller. Despite this it still more than doubled in size during 2014.

(more...)

Presentaties februari 2015

Presentaties februari 2015

Voor februari 2015 staan er diverse presentaties op de planning met name op het gebied van architectuur en agile:

Kennissesies agile architectuur bij Ordina

Op 5 februari organiseert Ordina een kennissessie over agile architectuur in de praktijk. Net als op het LAC zal ik een presentatie houden over de waarde van architectuur en time-to-market. Waarin ik laat zien dat juist architectuur waarde en snelheid kan bieden bij de time-to-market. Dit aan de (more...)

Making DevOps Business Driven – a service view

I've been doing a bit recently around DevOps and what I've been seeing is that companies that having been scaling DevOps tend to run into a problem: exactly what is a good boundary for a DevOps team? Now I've talked before about how Microservices are just SOA with a new logo, well there is an interesting piece about DevOps as well, its not actually a brand new thing.  Its an evolution and

Everybody Says “Hackathon”!

TLTR: MongoDB & Sage organized an internal Hackathon We use the new X3 Platform based on MongoDB, Node.js and HTML to add cool features to the ERP This shows that “any” enterprise can (should) do it to: look differently at software development build strong team spirit have fun! Introduction I have like many of you participated to multiple Hackathons where developers, designer and

Introducing OTBI-E

Today I was lucky enough to attend a pre-release workshop for OTBI-E (Oracle Transactional Business Intelligence for the Enterprise). The product is currently in controlled release but is going to be generally available very soon.

What is OTBI-E?

OTBI-E fills the gap between OTBI Standard (which is more operational in nature, reporting against the transactional database and for line managers to perform real-time BI) and the full OBIA (which is a fully-fledged data warehouse solution, delivered (more...)

Nantes MUG : Event #2

Last night the Nantes MUG (MongoDB Users Group) had its second event. More than 45 people signed up and joined us at the Epitech school (thanks for this!).  We were lucky to have 2 talks from local community members: How “MyScript Cloud” uses MongoDB by Mathieu Ruellan Aggregation Framework by Sebastien Prunier How “MyScript Cloud” uses MongoDB First of all, if you do not know MyScript I

Big Data and the importance of Meta-Data

Data isn't really respected in businesses, you can see that because unlike other corporate assets there is rarely a decent corporate catalog that shows what exists and who has it.  In the vast majority of companies there is more effort and automation put into tracking laptops than there is into cataloging and curating information. Historically we've sort of been able to get away with this

Security Big Data – Part 7 – a summary

Over six parts I've gone through a bit of a journey on what Big Data Security is all about. Securing Big Data is about layers Use the power of Big Data to secure Big Data How maths and machine learning helps Why its how you alert that matters Why Information Security is part of Information Governance Classifying Risk and the importance of Meta-Data The fundamental point here is that

Securing Big Data Part 6 – Classifying risk

So now your Information Governance groups consider Information Security to be important you have to then think about how they should be classifying the risk.  Now there are docs out there on some of these which talk about frameworks.  British Columbia's government has one for instance that talks about High, Medium and Low risk, but for me that really misses the point and over simplifies the

How to create a pub/sub application with MongoDB ? Introduction

In this article we will see how to create a pub/sub application (messaging, chat, notification), and this fully based on MongoDB (without any message broker like RabbitMQ, JMS, ... ). So, what needs to be done to achieve such thing: an application "publish" a message. In our case, we simply save a document into MongoDB another application, or thread, subscribe to these events and will received

Securing Big Data Part 5 – your Big Data Security team

What does your security team look like today? Or the IT equivalent, "the folks that say no".  The point is that in most companies information security isn't actually something that is considered important.  How do I know this?  Well because basically most IT Security teams are the equivalent of the nightclub bouncers, they aren't the people who own the club, they aren't as important as the

Securing Big Data – Part 4 – Not crying Wolf.

In the first three parts of this I talked about how Securing Big Data is about layers, and then about how you need to use the power of Big Data to secure Big Data, then how maths and machine learning helps to identify what is reasonable and was is anomalous. The Target Credit Card hack highlights this problem.  Alerts were made, lights did flash.  The problem was that so many lights flashed and

Securing Big Data – Part 3 – Security through Maths

In the first two parts of this I talked about how Securing Big Data is about layers, and then about how you need to use the power of Big Data to secure Big Data.  The next part is "what do you do with all that data?".   This is where Machine Learning and Mathematics comes in, in other words its about how you use Big Data analytics to secure Big Data. What you want (more...)

Securing Big Data – Part 2 – understanding the data required to secure it

In the first part of Securing Big Data I talked about the two different types of security.  The traditional IT and ACL security that needs to be done to match traditional solutions with an RDBMS but that is pretty much where those systems stop in terms of security which means they don't address the real threats out there, which are to do with cyber attacks and social engineering.  An ACL is only

Test locally first, then deploy to the cloud.

As recommended by Cheuk Chau on Medium, I read 'How to Develop a Daily Writing Habit’ - I’ll try to put it in practice here with my second advise - should have been my #1 advise:

Run a local copy of your application first, before to test it on your cloud instance.

While it’s simple to run directly on the Cloud, and we have a lot of samples available to get started, working (more...)