Categories

Developer / Photographer

Topics

angular angularjs AWS C# deduplication DevOps Digital Ocean encryption google analytics heroku itunes javascript Kubernetes Netstat Nginx optimization performance postgres ruby ruby on rails s3 security spotify
Web sockets with Digital Ocean Kubernetes

How To Use Web Sockets (Socket IO) With Digital Ocean Load Balancers And Kubernetes With Ingress Nginx

02/13/2020

Web sockets are awesome, although it’s not exactly new technology. You would assume that there is a wealth of information out there on the internet covering every possible use case you can imagine however recently I came across a problem that I had assumed would have a heavily documented solution online, but as it turns out, I found very little.

The problem I was trying to solve was running a multi server, web socket application (using Socket IO), within Kubernetes on Digital Oceans hosted K8S solution with a Digital Ocean load balancer attached to an Nginx Ingress controller. (That’s ingress-nginx, not nginx’s ingress controller)

This should be fine.. right? Well, there are some tricky gotchas here as I soon discovered that caught me out. I wasted hours tweaking configuration, so i’m writing this blog post so you don’t have to do the same.

Read More

Deduplicating Large Data With Rails

04/12/2017

Storing and retrieving large chunks of data from your database can be tricky if it’s not done correctly. What happens when you want to store a relatively large document or body of text in your database but that large chunk of text is likely to be identical for thousands of new records? You’re faced with the possibility of storing gigabytes of duplicated data that really doesn’t need to be there.

At Vue Pilot, we had this problem a lot. Thousands of duplicate records, with unique IDs but identical content just occupying disk space. It was pretty clear this wasn’t an optimal solution.
Read More

Uploading To S3 With AngularJS

08/14/2014

A little while back I found myself needing to handle file uploads without touching the application server. This is a common scenario for people using Heroku as they limit all requests to 30 seconds, which in theory sounds fine for most requests, but destroys any chance you have at directly handling file uploads within your application.

I restricted my allowed attachment size down to 2MB but still there were people in more remote parts of the world who were hitting that 30 second timeout. Heroku’s response is to simply not upload to the app server and instead go direct to something like AWS S3.aws

This got me thinking – can I upload a large file to Amazon S3 using the AWS-JS-SDK with JavaScript and work it into my existing AngularJS application? Read More

Simple Element Toggling With AngularJS

06/09/2014

Toggling UI elements is a chore that nobody enjoys. Back when your web apps were primarily jQuery or raw JavaScript you would need to wire up an event handler, catch the click event, determine the current visibility state and switch it. A fairly small job, but still an annoying one when you have a page full of these elements.

With Angular you can accomplish this without writing any code, simply by using two HTML attributes.
Read More

Resend Devise Confirmation Emails For Incomplete Accounts

12/19/2013

It’s slightly frustrating looking through your users table and seeing a bunch of accounts that signed up but never confirmed, only existing in this limbo land where you can’t quite count them as a conversion and they seem to have dropped off the face of the earth.

This rake task will look through your users table for unconfirmed devise accounts and one by one resend their confirmation emails. Read More

Using Gravatar With AngularJS

12/06/2013

Want to use Gravatar in your AngularJS app?
Use this simple directive to insert Gravatar images in your app.

Before you get started, you will need to make sure you have an MD5 hashed version of your users email address. Gravatar uses this hashed version of the email address in order to determine which avatar to display.

It’s simple in any back end language to generate this hash on page load, or send it down with your JSON model.

In Ruby

Digest::MD5.hexdigest(email_address)

In PHP

md5(email_address}

Now, Include this directive in your AngularJS application

 myApp.directive('gravatar', function() {
  return {
    restrict: 'AE',
    replace: true,
    scope: {
      name: '@',
      height: '@',
      width: '@',
      emailHash: '@'
    },
    link: function(scope, el, attr) {
     scope.defaultImage = 'https://somedomain.com/images/avatar.png';
    },
    template: '<img alt="{{ name }}" height="{{ height }}"  width="{{ width }}" src="https://secure.gravatar.com/avatar/{{ emailHash }}.jpg?s={{ width }}&d={{ defaultImage }}">'
  };
 });

You can change the “defaultImage” property to link to a default avatar to display when the user has no Gravatar.

Then simply drop the tag with attributes into your HTML

<span gravatar name="Bill Murray" width="50" email-hash="1726e726ac7163a6f"></span>

Serving Compressed Assets With Heroku and Rack-Zippy

09/29/2013

It’s often a little known fact that Heroku does by default , NOT serve the compressed version of your assets to the client  browser.

Often it’s all too easy to get lost in the magic that is Heroku slug compilation and the simplicity of a “git push heroku master” code deploy that we forget the basics of web development and end up serving our clients bloated CSS and JavaScript files, often at ridiculous sizes.

This is especially prominent of late with the rise of the “do it all” frameworks like Bootstrap, Foundation, AngularJS, Backbone JS etc.

Chances are, if you use any of these frameworks, you’re sending hundreds of kilobytes of code down to the browser on every request that is never even used, forcing your users to download the lot each time. Read More

Get Active Ports and Associated Process Names In C#

07/28/2013

Recently I found myself needing to find a way to determine what open and listening ports along with their associated running processes are currently active on a Windows machine using C#.

Some extensive Googling returned pretty dismal results.

Retrieving a list of open ports is simple.
Retrieving a list of running processes is simple.
A combination of the two is not.
There were a few external libraries, but it just seemed like overkill for something that should be a pretty simple task.

The solution was to just parse the output of a netstat -a -n -o command. The result from this code snip will return a list of “Port” objects which contain process names, port number and protocol as properties.
Read More

Heroku Postgres Row Limit Email Notifications

Heroku Postgres Row Limit Email Notifications

05/14/2013

So, you’ve got a side project hosted with Heroku, you only have basic database requirements so the 10,000 row limited Postgres dev plan seems like a good choice and could possibly be a free alternative for a descent amount of time until you need to upgrade.

Heck, at $35 a month per dyno, you need to try and save some money somewhere, right?

Take for example one of my side projects, it’s called NoteShred.com. It allows you to send password protected, encrypted notes to people over the internet with a unique URL and have the note automatically destroy its self after it’s been read.
The beauty of this application is that it is constantly deleting rows from the database every time a note is “shredded“, so a 10,000 row count is actually a descent amount of space for this application.

The problem is that 10,000 isn’t a huge number, and all it takes is one big spike of traffic and you’ve hit the limit.
Heroku provides a basic alerting service that will tell you once you’re at 7,000 records and again once you’ve hit the limit. You will have 24 hours to get your row count back under the limit before you will lose write access to your database. (Thanks to @hgmnz and @ctshryock for pointing this out)
More Info Here.

Unfortunately, This is not configurable and the cut off may come at a bad time, say if you’re on holidays or are not able to dive in and trim back the records before this 24 hour window expires.

The 7,000 row alert is nice how ever not terribly useful for applications like NoteShred where you have a slow creeping database and the difference between 7,000 records and 10,000 records is a large gap in time.
I am most likely going to disregard the 7,000 row alert email because I still have 3,000 left, the next email would be informing me that I have hit the limit and have 24 hours to get it back under control.

A configurable threshold with a series of alerts leading up to the cut off point would be more useful, or even just an alert closer to the limit, say at 9000 rows.

You want to keep an eye on this row count, you can do so using the Rails built in Mailer and a basic Rake task with this simple solution and be alerted when your application reaches a threshold nearing that 10,000 mark. You can easily customize this rake task to alert you at times when feel it’s more useful than the standard 7,000 and 10,000 alerts.

In a nutshell, this will:

  1. Check your row count every hour
  2. Email you if you’re over the threshold

Read More

Older Posts
© 2024 Cheyne Wallace | Developer / Photographer