Java: Format Date with java time

 This article was originally published on JRoller on March 11, 2017

An advantage of the DateTimeFormatter class from the java time package over the old DateFormat is that it is thread-safe. But if the only thing I have is a Date, how can I proceed?

The DateTimeFormatter's format method takes a TemporalAccessor as a parameter. Date has a toInstant() method that converts the Date to an Instant object that implements the TemporalAccessor interface. All that sounds pretty easy:

  Date date = new Date();
  DateTimeFormatter df = DateTimeFormatter.ISO_DATE_TIME;
  System.out.println(df.format(date.toInstant()));

Unfortunately, that does not work. You en up with the following error:

Exception in thread "main" java.time.temporal.UnsupportedTemporalTypeException: Unsupported field: Year
	at java.time.Instant.getLong(Unknown Source)
	at java.time.format.DateTimePrintContext$1.getLong(Unknown Source)
	at java.time.format.DateTimePrintContext.getValue(Unknown Source)
	at java.time.format.DateTimeFormatterBuilder$NumberPrinterParser.format(Unknown Source)

As it turns out, the only formatter that can be used with an Instant is DateTimeFormatter.ISO_INSTANT. And the output would be in UTC time zone and look like this: '2011-12-03T10:15:30Z'.

In fact, this is all very logical. The only information missing from a Date to know the correct time and date to print is the time zone. The old DateFormat class assumed you would want to use the current time zone. For java time, you have to explicitly sate it:

  Date date = new Date();
  DateTimeFormatter df = DateTimeFormatter.ISO_DATE_TIME;
  System.out.println(df.format(date.toInstant().atZone(ZoneId.systemDefault())));

Or you can also provide the time zone to the formatter:

  Date date = new Date();
  DateTimeFormatter df = DateTimeFormatter.ISO_DATE_TIME
    .withZone(ZoneId.systemDefault());
  System.out.println(df.format(date.toInstant()));

The toZone() method of Instant creates a ZonedDateTime object, from which you can easily convert to LocalDate, LocalTime or LocalDateTime. To convert back to a Date is always a matter of providing the missing information.

For instance, from a Localtime, you provide a date and a time zone:

  LocalTime now = LocalTime.now();
  Date date = Date.from(now.atDate(LocalDate.now()).atZone(ZoneId.systemDefault()).toInstant());

From a LocalDate, you need a time and a time zone:

  LocalDate now = LocalDate.now();
  Date date = Date.from(now.atStartOfDay().atZone(ZoneId.systemDefault()).toInstant());

Java: Create Temporary Files with NIO 2 on Linux

This article was originally published on JRoller on October 18, 2006 

We have an application that had problems with memory consumption. One type of object that we were keeping in memory was kept there only in case a user would start an administration client and connect it to our app. So instead of having it constantly in memory, we decided to serialize it into a temporary file that would be destroyed when the application would stop. When an admin would connect, we would read back the data and send it before removing it from memory again.

The code that created the temporary file looked more or less like that:

  Path dir = Paths.get("myDir");
  Path tempFile = Files.createTempFile(dir, "MyApp", null);

  OutputStream outputStream = Files.newOutputStream(tempFile, StandardOpenOption.DELETE_ON_CLOSE);

As you see, the creation of a temporary file goes in two steps. First, the createTempFile() method would create a unique name with the given prefix. Then, the DELETE_ON_CLOSE option would delete the file when it was closed, that is in our case when the application would exit.

All our tests conducted in development on Windows platform would work fine. However, our test team ran all their tests on a Linux platform, which was actually the target platform for production. And guess what? It failed. The file would be created fine, and data was written without any problem. But when an admin would connect, the application would return a FileNotFoundException. When we checked, indeed, the files were not there.

After some investigation, it turned out that the DELETE_ON_CLOSE option on Linux platform would delete the file immediately from the file system, while keeping a reference to it in the Output Stream so that we could still write to it. How did we fix it? By turning to a good old non-NIO method:

  Path dir = Paths.get("myDir");
  Path tempFile = Files.createTempFile(dir, "MyApp", null);
  tempFile.toFile().deleteOnExit();
  OutputStream outputStream = Files.newOutputStream(tempFile);

I have the feeling that 10 years later, delete on exit is still not included in NIO.

Java: Negating a Predicate

 This article was originally publixhed on JRoller on September 12, 2016

Often, when using Java streams, I try to replace my lambdas with Method References. For instance, when I have this code:

    mystream.filter(mystring -> mystring.isEmpty()) ...

I tend to replace it with that one:

    mystream.filter(String::isEmpty) ...

Except that this code is not really useful. I rarely filter my stream with empty Strings. Usually, I do quite the opposite:

    mystream.filter(mystring -> !mystring.isEmpty()) ...

Now with that one small symbol, I managed to make my day more difficult. What to do if I really want to use Method References? I can create a isNotEmpty() method, but I don't want to do that for all my methods that return a boolean. If I really insist, I can end up with this ugly code:

    mystream.filter(((Predicate<String>)String::isEmpty).negate()) ...

One way to make things nice again, is to create a helper method, like this one:

    public static <T> Predicate<T> not(Predicate<T> p) {
        return p.negate();
    }

Now, I can simply use it like this:

    mystream.filter(not(String::isEmpty)) ...

I heard that such a method might find its way into JDK 9. Someone heard anything more concrete?

It turned out that Predicate.not() was introduced in Java 11.

AWS: Cloudshell Python version mismatch

 Lately, I tried to install a Python library onto Cloudshell. I first checked the Python version:

$ python --version

Python 3.13.13

So I ran pip install, which deployed a version of my lib compatible with Python 3.13. The lib declares some entry points for setuptools, so that I can run some commands from the CLI. So I run those commands from the shell, but they fail with some errors about missing import.

After investigation, I found out that the command scripts started with a shebang like this one:

#!/usr/bin/python3

This is all normal, except that if I check the link, this is what I find:

$ ls -l /usr/bin/python3

lrwxrwxrwx. 1 root root 9 Apr 20 22:07 /usr/bin/python3-> python3.9

Someone forgot something? Or am I looking at it wrong?

AWS: The State of Account State

 In September 2025, AWS announced that the Account information in the Organizations Service will have a new State field to replace the Status field. Since that date, both fields are available for all Organizations operations, but the Status field is vowed to be removed on September 2026.

When you read such an announcement and you know your code is using the Status field, you project to review your code and update it. So we did quite immediately, but we could not see the new State field when executing our lambdas. So we postponed the update for later.

Recently, I had another look at the problem, and still could not see any State field appearing in lambdas. I tested some call to DescribeAccount within CloudShell, but the field was really there. So I decided to run the following lambda:

import boto3

import botocore

def lambda_handler(event, context):

    print("boto3:", boto3.__version__)

    print("botocore:", botocore.__version__)

    org_client = boto3.client("organizations")

    response = org_client.describe_account(AccountId="123456789012")

    print(response)

I was surprised by the result.

boto3: 1.40.4
botocore: 1.40.4

Those versions were released in August 2025, before the update. CloudShell in my test uses botocore 1.42.72, which is from March this year. When I notified AWS Support about it, they just told me to use a Layer with a more recent botocore included. How long should I keep this temporary workaround?

AWS: Duplicates in Search Provisioned Products

 Using our beloved boto3 library, we are looking for the list of all our Provisioned Products in Service Catalog.

sc_client = boto3.client('servicecatalog')

result = sc_client.search_provisioned_products(

    PageSize=20

)

I won't bore you with th code that loops over the result and perform the operation again if we have more than 20 products. But the strange thing, is that wherever we had more than the page size, some products were repeated in the other pages. Thinking of a bug in AWS Service Catalog, we reached out to the Support Team. This is their answer:

This is a known behavior with the SearchProvisionedProducts API when using the default relevance-based sorting. Because results are sorted by relevance, the ordering can shift slightly between paginated requests, which causes duplicates (or occasionally missed items) across pages.

Never heard of relevance-based sorting. Looking at the documentation, there is no mention of it:

SortBy

The sort field. If no value is specified, the results are not sorted. The valid values are arn, id, name, and lastRecordId.

 Then, Support Team is proposing a solution:

Adding SortBy='createdTime' gives the pagination a stable ordering, so the page token points to a consistent boundary between pages. No more duplicates should appear regardless of how many provisioned products you have.

It is interesting to note that 'createdTime' is not listed in the documentation either. We tried it and it works. So a hidden feature solves a known bug.

Collectors.toMap does not like null

 This article was originally published on JRoller on November 26, 2015

Some map accept null values, some don't. How do you know? You usually take a look in the javadoc. But what about maps created by streams through the Collectors.toMap? The javadoc does not say. So I tried out. I picked the following code:

 	List<String> player = Arrays.asList("Lebron", "Kobe", "Shaquille");
	List<String> team = Arrays.asList("Cleveland", "Los Angeles", null);
	
	Map<String, String> currentTeam = new HashMap<>();
	for (int i = 0; i < player.size(); i++) {
		currentTeam.put(player.get(i), team.get(i));
	}

Everything works as expected, it inserts the null value into my map. So I tried to convert it to streams (maybe not in the best way):

	Map<String, String> currentTeam = IntStream.range(0, player.size())
		.mapToObj(i -> i)
		.collect(Collectors.toMap(i -> player.get(i), i -> team.get(i)));

Here is what I get:

Exception in thread "main" java.lang.NullPointerException
	at java.util.HashMap.merge(HashMap.java:1216)
	at java.util.stream.Collectors.lambda$toMap$148(Collectors.java:1320)
	at java.util.stream.Collectors$$Lambda$6/149928006.accept(Unknown Source)
	at java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169)
	at java.util.stream.IntPipeline$4$1.accept(IntPipeline.java:250)
	at java.util.stream.Streams$RangeIntSpliterator.forEachRemaining(Streams.java:110)
	at java.util.Spliterator$OfInt.forEachRemaining(Spliterator.java:693)
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:512)
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:502)
	at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
	at Test.main(Test.java:23)

Is it using a kind of Map that does not accept null values? Let's check:

   public static <T, K, U>
    Collector<T, ?, Map<K,U>> toMap(Function<? super T, ? extends K> keyMapper,
                                    Function<? super T, ? extends U> valueMapper) {
        return toMap(keyMapper, valueMapper, throwingMerger(), HashMap::new);
    } 

Well, no. It uses a standard HashMap. In the stack, the Exception is thrown from the HashMap.merge() function. So let's have a look:

   @Override
    public V merge(K key, V value,
                   BiFunction<? super V, ? super V, ? extends V> remappingFunction) {
        if (value == null)
            throw new NullPointerException();
	...
    }

So the problem is not in the type of map but in the implementation of the merge. The javadoc for the HashMap merge() method says that the value parameter is "the non-null value to be merged with the existing value associated with the key". So yes, it says it in the Javadoc, but not where you would expect it.

By the way, if you really want to make it work with streams, you would have to supply your own collector (as an aside note, the same problem arises with the groupBy collector) :

	Map<String, String> currentTeam = IntStream.range(0, player.size())
		.mapToObj(i -> i)
		.collect(HashMap::new,
			(map, i) -> map.put(player.get(i), team.get(i)),
			HashMap::putAll);

Maybe I'll stick with the loop for this time.

Lambda Before Switch

 This article was originally published in JRoller on September 22, 2015.

We had this method that was doing the same thing several times in a row. Here is a simplified version:

void resolveAll() {
	a = resolveA();
	if (a == null)
		states.remove(State.A);
	else
		states.add(State.A);

	b = resolveB();
	if (b == null)
		states.remove(State.B);
	else
		states.add(State.B);

	c = resolveC();
	if (c == null)
		states.remove(State.C);
	else
		states.add(State.C);
} 

It's almost the same thing repeated three times (a bit more in the real life code), except for this method resolveX which is different each time, returning a different type of object. We needed to modify the code, but did not feel like having to repeat the change several times. So we resorted to a refactoring. But what to do with this resolveX method? Inner classes are really ugly, so we turned to a switch:

void resolveAll() {
	a = resolve(State.A);
	b = resolve(State.B);
	c = resolve(State.C);
} 
	
@SuppressWarnings("unchecked")
private <T> T resolve(State state) {
	T resolved = null;
	switch (state) {
		case A:
			resolved = (T) resolveA();
			break;
		case B:
			resolved = (T) resolveB();
			break;
		case C:
			resolved = (T) resolveC();
			break;
	} 

	if (resolved == null) {
		states.remove(state);
	}  else {
		states.add(state);
	} 
	
	return resolved;
} 

Not that much simpler than the original code. However, we are moving slowly to Java 8. With lambdas, inner classes do not look so ugly anymore:

void resolveAll() {
	a = resolve(State.A, this::resolveA);
	b = resolve(State.B, this::resolveB);
	c = resolve(State.C, this::resolveC);
} 

private <T> T resolve(State state, Supplier<T> resolver) {
	T resolved = resolver.get();
	
	if (resolved == null) {
		states.remove(state);
	}  else {
		states.add(state);
	} 
	
	return resolved;
} 

Thou Shall Close Thy Streams

 This article was originally posted on JRoller on March 12, 2015

You should always close your IO streams. Said like this, it sounds obvious. But in the light of some new Java 8 features, it took me some time to get around it.

I needed to write a small method for modifying a CSV file, basically change the last 1 of each line into a 0. Not really difficult. I would create a temporary file where I'll copy the original lines with the needed modification, then overwrite the original file with the one I created. Since I could use Java 8, I thought I would use a stream and lambdas. The code looked like this:

try (PrintWriter writer = new PrintWriter(Files.newBufferedWriter(tempFile))) {
	Files.lines(toConvert)
		.map(line -> line.replace(";1", ";0"))
		.forEach(line -> writer.println(line));
} 

Files.move(tempFile, toConvert, StandardCopyOption.ATOMIC_MOVE,
	StandardCopyOption.REPLACE_EXISTING);

Looks nice, except I had this weird java.nio.file.FileSystemException with the message "The process cannot access the file because it is being used by another process.". I was pretty sure that the only process using my file was my small program. So the problem was that the Files.lines() does not close the file. I found other references on the net to comfort my idea. So yes, I know, you can find it in the javadocs, and yes, the stream is autocloseable. So the way to go is the following:

try (Stream<String> reader = Files.lines(toConvert)) {
	reader.map(line -> line.replace(";1", ";0"))
		.forEach(line -> writer.println(line));
}

But to my defense, I'm not the only one having problems with the javadocs: https://bugs.openjdk.java.net/browse/JDK-8073923

Autoclose Lock

 This article was originally posted on JRoller on October 21, 2014

I was just wondering if there is a difference between the classic:

	lock.lock();
	try {
		 //I have the lock!
	}  finally {
		lock.unlock();
	} 

And the Autocloseable version:

	lock.lock();
	try (AutoCloseable auto = lock::unlock) {
		 //I have the lock!
	} 

Capital Date Mistake

 This article was originally pusblished on JRoller on April 10, 2014

Here is a small piece of code. Can you tell what it prints?

  SimpleDateFormat sdf = new SimpleDateFormat("YYYY-MM-dd");
  Calendar cal = Calendar.getInstance();
  cal.set(Calendar.YEAR, 2014);
  cal.set(Calendar.MONTH, Calendar.DECEMBER);
  cal.set(Calendar.DAY_OF_MONTH, 31);
  Date d = cal.getTime();
  System.out.println(sdf.format(d));

If you naively answered "2014-12-31", then I would tell you just this: you are really naive.

If you run this code under Java 6, you will get back an IllegalArgumentException, with the message "Illegal pattern character 'Y'". Now it might hit you that, indeed, the character for Year in a DateFormat is the lower case 'y'.

However, this code runs under Java 7, because the capital 'Y' was added to the DateFormat. But it does not stant for Year, but for Week Year. If, like me, you do not know what a Week Year is, here is the JavaDoc explanation:

A week year is in sync with a WEEK_OF_YEAR cycle. All weeks between the first and last weeks (inclusive) have the same week year value. Therefore, the first and last days of a week year may have different calendar year values.
For example, January 1, 1998 is a Thursday. If getFirstDayOfWeek() is MONDAY and getMinimalDaysInFirstWeek() is 4 (ISO 8601 standard compatible setting), then week 1 of 1998 starts on December 29, 1997, and ends on January 4, 1998. The week year is 1998 for the last three days of calendar year 1997. If, however, getFirstDayOfWeek() is SUNDAY, then week 1 of 1998 starts on January 4, 1998, and ends on January 10, 1998; the first three days of 1998 then are part of week 53 of 1997 and their week year is 1997.

In short, my exemple code will print "2015-12-31", because the last days of the year belong to a week of the following year.

Why don't I know my disk is full?

This article was originally posted on JRoller on March 14, 2014

We had this interesting problem lately, that an application had a corrupted file after a disk got full. The strange thing was that the app did not know that the disk was full. We were expecting an IOException, but no trace of it nowhere. Moreover, the app went on running, and resumed writing in the file when someone deleted a big file from the disk.

While looking on the net for how to know when a disk is full, we got this code strip on StackOverflow:

  FileOutputStream fos = ...;
  fos.write("hello".getBytes());
  fos.getFD().sync();
  fos.close();

This forces the OS to force synchronization of the file with its internal buffer, and throws a SyncFailedException when the disk is full. However, this has horrible performance. So we kept looking for our IOException. After a bit of investigation, we found out the culprit: PrintWriter.

The PrintWriter class can be very useful, since it wraps common print methods around a stream. Also, it automatically creates a BufferedWriter, as you can see in this constructor:

    public PrintWriter(File file) throws FileNotFoundException {
        this(new BufferedWriter(new OutputStreamWriter(new FileOutputStream(file))),
             false);
    }

Another features of the PrintWriter, and now we get to the problem, is that most of its methods do not throw any Exception. Here is an example of a typical PrintWriter method:

    public void write(char buf[], int off, int len) {
        try {
            synchronized (lock) {
                ensureOpen();
                out.write(buf, off, len);
            } 
        } 
        catch (InterruptedIOException x) {
            Thread.currentThread().interrupt();
        }
        catch (IOException x) {
            trouble = true;
        } 
    } 

You can see that all IOExceptions are caught, and a boolean is set to true. The only way to know that something went wrong is to call the method checkError(). This only tells you that something went amiss, but not what, since you lost the exception.

    public boolean checkError() {
        if (out != null) {
            flush();
        } 
        if (out instanceof java.io.PrintWriter) {
            PrintWriter pw = (PrintWriter) out;
            return pw.checkError();
        }  else if (psOut != null) {
            return psOut.checkError();
        } 
        return trouble;
    } 

Notice how it checks if it wraps another PrintWriter or a PrintOutputStream, because it knows that they swallow the Exception.

At the end, we replaced the PrintWriter with a simple FileWriter, because we did not need the PrintWriter in our case. We only needed a Writer to give to our CSV exporter library, which already wrapped it in a BufferedWriter. Now, IOExceptions are nicely bubbling up, and we know when our disk is full. 

AutoCloseable when not available

 This article was originally posted on JRoller on January 14, 2014

On his blog, Brian Oxley shows a neat way in JDK 8 to use non AutoCloseable object having a close-like method in a try with resources construct, using the new method references. For those of us who are still stuck with JDK 7 for some time, here is the (uglier) equivalent code:

	final Foo foo = new Foo();
	try (AutoCloseable ignore = new AutoCloseable() {
		@Override
		public void close() {
			foo.close();
		} 
	}) {
		foo.doFoo();
	} 

Permutations and Combinations

 This article was originally posted on JRoller on October 22, 2013

A simple piece of code for calculating all permutations of a given word:

    public static List permutations(String s) {
        List list = new ArrayList<>();
        permutations(list, s, "");
        return list;
    }

    private static void permutations(List list, String from, String to) {
        if (from.isEmpty()) {
            list.add(to);
        }
        else {
            for (int i = 0; i < from.length(); i++) {
                permutations(list,
                        new StringBuilder(from).deleteCharAt(i).toString(),
                        to + from.charAt(i));
            }
        }
    }

The code for permutations of N elements out of a word is a simple modification of the previous program:

    public static List permutationsN(String s, int n) {
        List list = new ArrayList<>();
        permutationsN(list, s, "", n);
        return list;
    }

    private static void permutationsN(List list, String from, String to, int n) {
        if (to.length() == n) {
            list.add(to);
        }
        else {
            for (int i = 0; i < from.length(); i++) {
                permutationsN(list,
                        new StringBuilder(from).deleteCharAt(i).toString(),
                        to + from.charAt(i),
                        n);
            }
        }
    }

For Combinations, that is when order is not important, another small modification does the trick:

    public static List combination(String s, int n) {
        List list = new ArrayList<>();
        combination(list, s, "", 0, n);
        return list;
    }

    private static void combination(List list, String from, String to, int index, int n) {
        if (to.length() == n) {
            list.add(to);
        }
        else {
            for (int i = index; i < from.length(); i++) {
                combination(list,
                        new StringBuilder(from).deleteCharAt(i).toString(),
                        to + from.charAt(i),
                        i,
                        n);
            }
        }
    }

Modal vs Always on Top

This article was originally posted on JRoller on Auguts 2, 2013

Here is an interesting piece of code:

      public static void main(String[] args) throws Exception {
          JFrame alwaysOnTopFrame = new JFrame(\"Always on top Frame\");
          alwaysOnTopFrame.setAlwaysOnTop(true);
          alwaysOnTopFrame.setDefaultCloseOperation(JFrame.DISPOSE_ON_CLOSE);
          alwaysOnTopFrame.setSize(500, 500);
          alwaysOnTopFrame.setVisible(true);
          alwaysOnTopFrame.setLocationRelativeTo(null);

          JOptionPane.showConfirmDialog(null, \"coucou\");
      }

It displays a simple frame, with the "Always on Top" activated, then opens a modal dialog. What will happen? Which one will prevail and be in front of the other?

As it turns out, if you try to run this code, you will see the Frame hiding the modal dialog box, although the application waits for you to answer the dialog first. What will appear to the average user is that there is a frozen window which does not react to anything. Impossible to move it or close it, until you guess that there is a dialog box behind it.

Of course, if you know it, you can discard the dialog box using Enter or Escape key, or the Alt+F4 combination, or access the dialog box menu through Alt+Space and move it around using the arrow keys. But the average user won't know it.

What is the solution? There is actually an option that you can set on the Frame, so that it will still respond to user actions even though there are modal dialogs on the screen. It is called Modal Exclusion Type. Try adding the following line:

  alwaysOnTopFrame.setModalExclusionType(ModalExclusionType.APPLICATION_EXCLUDE);

When is a SwingWorker really done?

This article was originally posted on JRoller on April 25, 2013

Imagine you have a SwingWorker running, and you would like to know if it has terminated its job. You notice that it has a isDone() method, and say to yourself: "great! I can use it!". But are you sure you know what it is really doing? Are you sure that all your work will be over when it returns 'true'? You would say: "of course, it is called isDone". Well, let's have a closer look.

When you create a SwingWorker, it creates a Callable, and wraps it in a Future:

        Callable callable =
                new Callable() {
                    public T call() throws Exception {
                        setState(StateValue.STARTED);
                        return doInBackground();
                    }
                };

        future = new FutureTask(callable) {
                       @Override
                       protected void done() {
                           doneEDT();
                           setState(StateValue.DONE);
                       }
                   };

The Future will be executed by a ThreadPool (of maximum 10 threads). The doInBakground() is called, returning a value that will be used by the Future for its set() method. From this point onwards, calling the isDone() method will yield true. And this is even before the Future's done() method is called. So isDone() only means that doInBackground() terminated. Not what I was thinking.

OK, so why not use the getState() method. I see clearly that it is called by the Future right after the call to doneEDT(). But let's have a look at the doneEDT() method:

    private void doneEDT() {
        Runnable doDone =
            new Runnable() {
                public void run() {
                    done();
                }
            };
        if (SwingUtilities.isEventDispatchThread()) {
            doDone.run();
        } else {
            doSubmit.add(doDone);
        }
    }

As you can see, if we are already in the EDT, we execute immediately the method. But we know that we are started from a ThreadPool, so this is not the case. The doSubmit, surprisingly, uses a Timer and schedules the task in 30ms. Not a SwingUtilities.invokeLater() as I expected, but the result is the same: the getState() will return DONE before we even start the done() method. The PropertyChange are notifying you of the state change. So basically, you are left to yourself.

EDT Freeze Detector

 This article was originally posted on Jroller on April 2, 2013

It is always the same scenario: the support team contacts us with a problem of a "frozen GUI". They send us the logs so that we can investigate, but of course the logs do not show anything. The user already restarted its GUI, so when we ask the support to perform a jstack, it is already too late. Quite often, it is not even due to a deadlock, but to a long operation that should not be done in the EDT. Since the only thing we have to investigate is the logs, I decided to write an EDT Freeze Detector that would log any operation monopolizing the EDT for more than 10 seconds. Here is the code:

import java.awt.AWTEvent;
import java.awt.EventQueue;
import java.awt.Toolkit;
import java.util.Timer;
import java.util.TimerTask;

public class FreezeDetector extends EventQueue{
    private static final long FREEZE_TIMER_PERIOD = 10000L;

    private volatile AWTEvent currentEvent;
    private volatile Thread eventDispatchThread;

    private FreezeDetector() {
        Timer timer = new Timer("Freeze Detector", true);
        timer.schedule(new FreezeTimerTask(), FREEZE_TIMER_PERIOD, FREEZE_TIMER_PERIOD);
    }

    public static void installFreezeDetector() {
        Toolkit.getDefaultToolkit().getSystemEventQueue().push(new FreezeDetector());
    }

    @Override
    protected void dispatchEvent(AWTEvent event) {
        eventDispatchThread = Thread.currentThread();
        currentEvent = event;

        try {
            super.dispatchEvent(event);
        }
        finally {
            currentEvent = null;
        }
    }

    private class FreezeTimerTask extends TimerTask {
        private AWTEvent lastEvent;

        @Override
        public void run() {
            if (lastEvent != null && lastEvent == currentEvent) {
                printStack();
            }

            lastEvent = currentEvent;
        }

        private void printStack() {
            StackTraceElement[] stackTrace = eventDispatchThread.getStackTrace();
            StringBuilder sb = new StringBuilder();
            sb.append("Freeze detected on EDT:");

            for (StackTraceElement stackElement : stackTrace) {
                sb.append(stackElement.toString()).append('\n');
            }

            //use your favorite logger
            System.out.println(sb);
        }
    }
}

AWS: Step functions can keep only Lambda payloads

 When running an AWS Lambda in synchronous mode from a Step Function, you will get the Lambda's return value in the output's Payload part. But unfortunately, you might also get some mostly useless information as well:

{

    "ExecutedVersion": "$LATEST",

    "Payload": {

        "result": "value"

    },

    "SdkHttpMetadata": {},

    "HttpHeaders": {}

}

When browsing your Step Functions's output, that's a lot of noise. So to keep only the lambda's payload part, you can add this line in your task definition:

    "ResultSelector": { "Payload.$": "$.Payload" }

Keep only the good stuff!

AWS: Physical Resource ID in Custom Resources

Originally, Custom Resources in Cloudformation were designed for wrapping AWS resources that are not yet supported by the Cloudformation service into a Lambda. However, we often use them for other purposes:

• Retrieve information from other resources (like the data in terraform)
• Trigger some actions
• Implement some logic

In most of those cases, we do not care about resource deletion. But we are often surprised by calls from Cloudformation to delete the resource. The reason is the misunderstanding or the misuse of the Physical Resource ID. And the origin of this, for me, comes from a bad design choice on AWS part.

Let's have a look at the way the cfnresponse module is written. 

def send(event, context, responseStatus, responseData, physicalResourceId=None, noEcho=False, reason=None):
    responseUrl = event['ResponseURL']
    responseBody = {
        'Status' : responseStatus,
        'Reason' : reason or "See the details in CloudWatch Log Stream: {}".format(context.log_stream_name),
        'PhysicalResourceId' : physicalResourceId or context.log_stream_name,
        'StackId' : event['StackId'],
        'RequestId' : event['RequestId'],
        'LogicalResourceId' : event['LogicalResourceId'],
        'NoEcho' : noEcho,
        'Data' : responseData
    }

There are 2 bad choices:

•  The Physical Resource ID parameter is optional. It makes you think that it is not important. That if you don't set it, some default behavior will handle it correctly for you. 
• The default value is random. Even worse, it is not consistently random. It is set to the log stream name, that changes on each Lambda cold start.

That means that most of the time, your Physical Resource ID will change on each call, except if you trigger it several times in a row. And this change of Physical Resource ID is the one that triggers the call to the delete part of your Lambda.

You can imagine that your Resource behaves like an EC2. If you modify a tag, your instance will keep its ID. But if you change its VPC, a new instance will be created, with a new ID. In that case, the old instance must be deleted. You can consider the Physical Resource ID to be like the instance ID. You want to decide, based on which parameter was modified, if the old Resource must be kept or deleted. 

Which means that in most cases, you do not want your Physical Resource ID to change. So the default behavior is wrong. It will lead to:

• Have your Lambda called for deletion for no reason.
• Can cause accidental calls that you don't expect. We had the case when an S3 bucket was deleted in production because someone added a parameter to the Custom Resource.
• Makes you write some useless code to avoid to call the Lambda when you don't expect it, like checking that your Cloudformation stack is really deleting the Resource. 

So the correct behavior is to always set the Physical Resource ID. And usually to a constant value:

cfnresponse.send(event, context, cfnresponse.SUCCESS, responseData, "ConstantPhysicalID")

What about the legacy code? Those old Custom Resources that already have a Physical Resource ID set to the log stream name? The good thing is that the previously set Physical Resource ID is sent to the Lambda in the event parameter. So you can simply set it back to its previous value:

physicalId = event["PhysicalResourceId"]

cfnresponse.send(event, context, cfnresponse.SUCCESS, responseData, physicalId)

AWS: Why Serverless Macro for Cloudformation always packages?

I'm using the Serverless macro quite a lot in my Cloudformation templates. It is very practical that you point a Lambda content to a local folder. Then Cloudformation packages the whole content into a Zip file and uploads it to an S3 Bucket.

However, it happens that I use the Serverless macro for some other feature, like generating the Event Rule that trigger my Lambda for instance. In some cases, my Lambda code can be already packaged in a Container on ECR, or even inlined. In those cases, I don't need any packaging.

What I noticed, is that Cloudformation is still packaging something. I downloaded the packaged Zip and checked its content. I could find the complete folder from the Cloudformation template location. For one template that was stored in the root of my source code, it packaged the complete application!

Does someone know why is that? Is there a reason for packaging when a Lambda is only inlined? Is there a way to tell Cloudformation to avoid packaging? Is it a bug?