Monday, January 23, 2023

JComboBox breaks on Exception in ActionListener

This article was posted originally on JRoller on the 7/9/2011. 

We had a bug reported to us the other day, when one combobox suddenly stopped working. You could select any item, the other components would not react to the selection. Looking through the logs, we noticed a nice NullPointerException, but only one. It seems like the first exception in our ActionListener was breaking something in our combobox. So I looked at the source code of JComboBox, and here is what I found: 
    protected void fireActionEvent() {
    if (!firingActionEvent) {
    firingActionEvent = true;
    // Loop through Listeners
    //call to listener.actionPerformed(e)
    firingActionEvent = false;
    }    
    }


Notice the firingActionEvent boolean, that is here to prevent an infinite loop of event. If an exception occurs in the listener, the boolean firingActionEvent remains at true, and no action will ever be processed. I filed a bug report to Oracle, so that they add a try/finally block to ensure the boolean goes back to false at the end of the method.

Looking a bit further in the code, I could find the same pattern applied in the setSelectedItem() method:

     // Must toggle the state of this flag since this method    
    // call may result in ListDataEvents being fired.
    selectingItem = true;
    dataModel.setSelectedItem(objectToSelect);
    selectingItem = false;

 I guess it would be a good idea to apply the same fix here.

Sunday, January 22, 2023

Found Archives of my Old Blog!

Eons ago, I started blogging about Java on JRoller.com. Unfortunately, the company running the site disappeared. And being really lazy, I lost all my articles. About 5 years ago, I restarted blogging on Blogger, feeding first my articles from my talk in Devoxx "The Awakening of the Threads", then adding new material as it came, mainly on Python and AWS. But today, completely by accident, I found that there is an archive of 73 of my old blog articles dating from 2011 to 2017 on rssing. So I will try to re-post them here as time allows, travelling into my past, and rediscovering old java stuff.

Monday, January 2, 2023

Warnings on SQS endpoint URL while mocking with moto

 We have many unit tests using moto library to mock calls to boto3 to access AWS services.They all work fine, except for a warning on the use of this fixture:

@fixture(scope='function')
def sqs():
    with mock_sqs():
        yield boto3.client('sqs', region_name=AWS_REGION)

This is one of the correct ways of using moto mocks. But only this mock on SQS shows the following warning:

FutureWarning: The sqs client is currently using a deprecated endpoint: eu-central-1.queue.amazonaws.com. In the next minor version this will be moved to sqs.eu-central-1.amazonaws.com. See https://github.com/boto/botocore/issues/2705 for more details.

 Now, our Sonar is considering these warnings as errors, and leads our build to fail. As explained by the issue, the warning comes from the use of a deprecated attribute called sslCommonName. I didn't dive into the moto library, but there might be a way to avoid the use of this attribute. The easisest way to stop using it, is to add this environment variable to our test code:

import os
os.environ['BOTO_DISABLE_COMMONNAME'] = 'true'

This skips completely the use of the deprecated attribute.

Friday, December 30, 2022

Dict's get has a Default Value

 I found this pattern in several places in our Python code base:

state = "on"
if "State" in data and data["State"] == "off":
    state = "off"

The state here can have two values: on or off. So all this code can be pretty much replaced with this one line:

state = data.get("State", "on")

If the state is missing, it will get the value "on". Of course, you might say that it is not completely equivalent, since anything but "off" would be transformed into "on" in the first code. But then, this code is usually followed by this condition:

if state != "off":

I would have rather replaced the state variable with a boolean, and have a code like this one instead:

state_on = data.get("State") != "off"
if state_on:

If the state is missing, the get would return None, which is also different from "off".

Thursday, October 6, 2022

WTF: Enumeration Galore

 I found some Python code from a guy that used lots of enumeration. He probably copied the code from somewhere and did not understand it well. At the end, we have some pretty confusing code. For example, he wanted to add to a list of trusting accounts some accounts from another list, avoiding duplication. Here is his code:

for i, index in enumerate(account_list):
    if len([a for a in trusting_accounts if a == account_list[i]]) == 0:
        trusting_accounts += [account_list[i]]

The enumerate function returns both the index and the item in the list. However, the fact that he named them i and index shows me that he had no idea what the second part was about. He actually never used it, always referring to the index confusingly called 'i'.

The if condition is even more confusing. It triggers only if the list of accounts having the same name as the current account in the loop is empty. Simply said, it triggers if the account is not already in the list. The corrected version of the code is here:

for account in account_list:
    if account not in trusting_accounts:
        trusting_accounts.append(account)

Of course, this pattern was used in a lots of place in his code. The function for removing a list of accounts from the trusting accounts for instance looks like this:

for i, index in enumerate(account_list):
    for ii, iindex in enumerate(trusting_accounts):
        if account_list[i] == trusting_accounts[ii]:
            trusting_accounts.pop(ii)

There is an inner enumeration to find the index of the account to remove. He probably didn't know that there is a remove function in the list doing all the work. Here is the corrected code:

for account in account_list:
    if account in trusting_accounts:
        trusting_accounts.remove(account)

Much clearer...

Thursday, September 22, 2022

AWS: Unblock CloudFormation stacks from UPDATE_ROLLBACK_FAILED state

 In our project, we have lots of AWS accounts. And we are in charge of deploying base resources in each of them. To do this, we use CloudFormation to deploy several stacks. Actually several stacks nested into one master stack.

One problem we have is that our customers do not always update their stacks to the latest version. Also, when a stack fails to update, they sometimes let it rollback, and do not care to ask for a fix. Of course, this summer, there was even less updates, people being on vacation. And also no release, since we felt there would be nobody to deploy it. So we end up this September with a larger release than usual.

The result of all this is that we found ourselves staring at lots of accounts with a stack entering UPDATE_ROLLBACK_FAILED state. The reason? Deprecation. As it happened, AWS decided to deprecate Python 3.6, and also a couple of Policies (AWSConfigRole, AWSCloudTrailReadOnlyAccess). Of course we updated our stacks with the correct values for Python and the replacement policies some time ago. But as the stacks were not always up to date, and there were some issues while updating to the new release, many stacks started to rollback. And since some of the rollbacked values were deprecated, the rollbacks failed.

When you are in that case, you have two choices. The first one is to delete everything and redeploy. We tried it on one account, and it was really painful. Too many dependencies and manual actions. The second one is the one advised to us by the AWS support itself: continuing rolling back. When you continue a rollback, you have the possibilities to skip some resources. In our case, we needed to skip all lambdas using Python 3.6, and all roles using the deprecated policies.

We tried it manually in the AWS Console, and there is one caveat: you can select resources from nested stacks, but not resources from stacks nested into nested stacks. Since we have many accounts to update, and many resources to rollback, we decided to script the whole process.

We thought it will be simple: using Python and boto3, we list all the resources in our stack, recursively entering nested stacks, and filtering all lambdas and roles. We ran into several problems:

  • You cannot skip resources that are not in a failed stack
  • You cannot skip resources that are not in a failed state
  • You cannot skip resources that are in a failed state because CloudFormation cancelled the update
  • Once you run rollback with skipped resources, CloudFormation discovers new failing resources, so you have to iterate until all is fine, or the list of resources to skip does not change between two iterations.
  • Name of resources in nested stack are <nested_stack_name>.<resource_logical_id>. Even for resources in several level of nested stack, you still use the same pattern, giving only the name of the direct parent stack.
  • Waiting for a rollback to complete will throw an exception if the rollback fails.

This is the script that helps turning a stack from UPDATE_ROLLBACK_FAILED to UPDATE_ROLLBACK_COMPLETE:

import boto3

BLOCKABLE_RESOURCES = [
    "AWS::Lambda::Function",
    "AWS::IAM::Role",
]

STACK_NAME = "MyStack"


def get_stack_status(cf_client, stack_name):
    response = cf_client.describe_stacks(StackName=stack_name)
    return response["Stacks"][0]["StackStatus"]


def find_blocking_resources(cf_client, stack_name, parent, resources):
    response = cf_client.describe_stack_resources(StackName=stack_name)

    if (
        parent
        and get_stack_status(cf_client, stack_name) != "UPDATE_ROLLBAK_FAILED"
    ):
        return

    for resource in response["StackResources"]:
        if (
            resource["ResourceType"] == "AWS::CloudFormation::Stack"
            and resource["ResourceStatus"] == "UPDATE_FAILED"
        ):
            nested_name = resource["PhysicalResourceId"].split("/")[1]
            find_blocking_resources(
                cf_client, nested_name, nested_name + ".", resources
            )
        elif (
            resource["ResourceType"] in BLOCKABLE_RESOURCES
            and resource["ResourceStatus"] == "UPDATE_FAILED"
            and resource.get("ResourceStatusReason")
            != "Resource update cancelled"
        ):
            resource.append(parent + resource["LogicalResourceId"])


cf_client = boto3.client("cloudformation")

status = get_stack_status(cf_client, STACK_NAME)

if status != "UPDATE_ROLLBACK_FAILED":
    print("Nothing to unblock. Exiting")
    exit(0)

resources = []
previous_resources = ["DUMMY"]
waiter = cf_client.get_waiter("stack_rollback_complete")

print("Starting unblocking process")
while status != "UPDATE_ROLLBACK_FAILED" and previous_resources != resources:
    previous_resources = resources
    resources = []
    find_blocking_resources(cf_client, STACK_NAME, "", resources)
    print("Skipping ", resources)

    cf_client.continue_update_rollback(
        StackName=STACK_NAME, ResourcesToSkip=resources
    )
    try:
        waiter.wait(StackName=STACK_NAME)
    except Exception as err:
        print(err)

    status = get_stack_status(cf_client, STACK_NAME)

print("Final stack status:", status)

Of course, once this is done, you still have to fix the stacks and run an update.

Thursday, August 25, 2022

Modifying AWS resource for testing

 If you are writing unit tests using boto3 and moto, you might want to modify a value in a resource. I had this case, for instance, where I wanted to set the state of an AMI to pending for a test. Using moto, AMIs are always created in the available state. However, if you try to do it this way, it will fail:

ec2 = boto3.resource("ec2")
image = ec2.Image(AMI_ID)
image.state = "pending"

You will end up with this message:

AttributeError: can't set attribute 'state'

It is possible to modify the resource object. However, it will just modify this object instance, the model itself will not be updated. So if you use the resource as a parameter to a function, it will work. But if the function retrieves its data with boto3, it will get the original value.

Here is my solution:

image.meta.data.update({"State": "pending"})

That helped in my case.