JIRA, Python, Technical

Jython: Timezone Manipulation and Meeting Invitations vs Outlook

Link: http://dannyman.toldme.com/2013/09/23/jython-timezone-manipulation-and-meeting-invitations-vs-outlook/

As part of a project at work I’ve built some Jython code that builds iCalendar attachments to include meeting invitations for scheduled maintenance sessions. Jython is Python-in-Java which takes some getting used to but is damned handy when you’re working with JIRA. I will share a few anecdotes:

1) For doing date and time calculations, specifically to determine locale offset from UTC, you’re a lot happier calling the Java SimpleDateFormat stuff than you are dealing with Python. Python is a beautiful language but I burned a lot of time in an earlier version of this code figuring out how to convert between different time objects for manipulation and whatnot. This is not what you would expect from an intuitive, weakly-typed language, and it is interesting to find that the more obtuse, strongly-typed language handles time zones and it just fricking works.

Here is some sample code:

from java.text import SimpleDateFormat
from com.atlassian.jira.timezone import TimeZoneManagerImpl
tzm = TimeZoneManagerImpl(ComponentManager.getInstance().getJiraAuthenticationContext(),
# df_utc = DateFormat UTC
# df_assignee = DateFormat Assignee
df_utc = SimpleDateFormat("EEE yyyy-MM-dd HH:mm ZZZZZ (zzz)")
df_assignee = SimpleDateFormat("EEE yyyy-MM-dd HH:mm ZZZZZ (zzz)")
tz = df_utc.getTimeZone()
issue_dict['Start_Time_text'] = df_utc.format(start_time.getTime())
issue_dict['Start_Time_html'] = df_utc.format(start_time.getTime())
if df_utc != df_assignee:
    issue_dict['Start_Time_text'] += "\r\n                   "
    issue_dict['Start_Time_text'] += df_assignee.format(start_time.getTime())
    issue_dict['Start_Time_html'] += "<br />"
    issue_dict['Start_Time_html'] += df_assignee.format(start_time.getTime())
# Get TimeZone of Assignee
# Start Time in Assignee TimeZone

Since our team is global I set up our announcement emails to render the time in UTC, and, if it is different, in the time zone of the person leading the change. For example:

Start Time: Mon 2013-09-23 23:00 +0000 (UTC)
Mon 2013-09-23 16:00 -0700 (PDT)

2) I was sending meeting invitations with the host set to the assignee of the maintenance event. This seemed reasonable to me, but when Mac Outlook saw that the host was set, it would not offer to add the event to the host’s calendar. After all, all meeting invitations come from Microsoft Outlook, right?! If I am the host it must already be on my calendar!!

I tried just not setting the host. This worked fine except now people would RSVP to the event and they would get an error stuck in their outboxes.

So . . . set the host to a bogus email address? My boss was like “just change the code to send two different invitations” which sounds easy enough for him but I know how creaky and fun to debug is my code. I came upon a better solution: I set the host address to user+calendar@domain.com. This way, Outlook is naive enough to believe the email address doesn’t match, but all our software which handles mail delivery knows the old ways of address extension . . . I can send one invitation, and have that much less messy code to maintain.

from icalendar import Calendar, Event, UTC, vText, vCalAddress
# [ . . . ]
event = Event()
# [ . . . ]
# THIS trick allows organizer to add event without breaking RSVP
# decline functionality.  (Outlook and its users suck.)
organizer_a = assignee.getEmailAddress().split('@')
organizer = vCalAddress('MAILTO:' + organizer_a[0]+ '+calendar@' +
organizer.params['CN'] = vText(assignee.getDisplayName() + ' (' + assignee.getName() + ')')
event['organizer'] = organizer

You can get an idea of what fun it is to build iCalendar invitations, yes? The thing with the parentheses concatenation on the CN line is to follow our organization’s convention of rendering email addresses as “user@organization.com (Full Name)”.

3) Okay, third anecdote. You see in my first code fragment that I’m building up text objects for HTML and plaintext. I feed them into templates and craft a beautiful mime multipart/alternative with HTML and nicely-formatted plaintext . . . however, if there’s a Calendar invite also attached then Microsoft Exchange blows all that away, mangles the HTML to RTF and back again to HTML, and then renders its own text version of the RTF. My effort to make a pretty text email for the users gets chewed up and spat out, and my HTML gets mangled up, too. (And, yes, I work with SysAdmins so some users actually do look at the plain text . . .) I hate you, Microsoft Exchange!

Feedback Welcome

JIRA, Python, Technical

Jython Validator Cookbook

Link: http://dannyman.toldme.com/2012/09/25/jython-validator-cookbook/

Building on a previous post to validate user time tracking, a few “cookbook” scripts that may be handy to you or me in the future.


Validate User Time Tracking

Somewhat elaborate: enforce that time worked has been logged, except under certain circumstances. See original post.

import com.atlassian.jira.issue.worklog.Worklog
from com.atlassian.jira import ComponentManager
# Time Already Logged
timespent = issue.getTimeSpent()
# Time Logged via current screen
    timelogged = dict(issue.getModifiedFields())['worklog']
    timelogged = False
# Duplicate Issue?  It is as good as logged!
resolution = issue.getResolution()
if resolution['name'] == "Duplicate":
    timelogged = True
if resolution['name'] == "Self Corrected":
    timelogged = True
# Nagios likes to close tickets, but doesn't get paid
user = ComponentManager.getInstance().getJiraAuthenticationContext().getUser()
if user.getName() == "nagios":
    timelogged = True
if timespent < = 0 and timelogged == False:
    result = False
    description = "Please log the time you spent on this ticket."

Assign Unassigned Issue to User

This helps make sure tickets get assigned.

# -*- coding: UTF-8 -*-
from com.atlassian.jira import ComponentManager
assignee = issue.getAssignee()
user = ComponentManager.getInstance().getJiraAuthenticationContext().getUser()
if not issue.getAssignee():

Validate Custom Field Value

We have a particular custom field which can be set UNKNOWN by the Reporter, but which should be cleaned up by the Assignee.

from com.atlassian.jira import ComponentManager
cfm = ComponentManager.getInstance().getCustomFieldManager()
product = issue.getCustomFieldValue(cfm.getCustomFieldObject('customfield_12345'))
if product == 'UNKNOWN':
result = False
description = "Please set CUSTOM_FIELD value appropriately."

Feedback Welcome

Python, Sundry, Technology

PYCON Starts March 7th!

Link: http://dannyman.toldme.com/2012/02/08/derr/

From an email to colleagues:

Yesterday I got Dr Sick Wife to drop me off at the Santa Clara Convention Center so Mr Sicker Light-Headed Husband wouldn’t miss any PYCON. After an awkward twenty minutes of asking people there for the lighting and LED conference where the Python was, I checked my smart phone and noted that … PYCON is *MARCH* 7th.

So I took the light rail home and told $BOSS I was on PTO (well, I call it MLK day due to Puppet Training) I then slept a lot, and did other things sick people do that don’t bear repeating in a professional context, and watched Dr Who save the Earth on TV, slept some more, and I am feeling way better today, which means I feel regular sick, not super sick.

So, I’ll be WFH today. Trust me, whatever this is, you’re lucky to miss out! I don’t normally get sick so this is a novel experience … I’ll likely be seen in the office next week, though if I’m coughy or sneezy I’ll keep that train wreck at home, because, as you might gather, you don’t want a piece of this!

If you’re attending PyCon, I look forward to seeing you there … next month! Hopefully I won’t be light-headed!

Feedback Welcome

Python, Technical

Python: List to English String

Link: http://dannyman.toldme.com/2010/06/30/python-list-comma-comma-and/

Well, I am working on extending a Django application to add log entries to the django.contrib.admin.models.LogEntry which may be fodder for another post, but while composing a change_message I wanted to convert a list of “changes” into a string like “Changed this thing, that thing, and that other thing.”

Here is what I have got, and since it is Python I bet $1 that someone will comment with a better way. (I couldn’t figure a good search query for seeking the answer so I had to use my brain.)

if len(updated_list) > 1:
rs = "Changed " + ", ".join(map(str, updated_list[:-1])) + " and " + updated_list[-1] + "."
rs = "Changed " + updated_list[0] + "."
>>> updated_list = ['just one thing']
>>> print "Changed " + updated_list[0] + "."
Changed just one thing.
>>> updated_list = ['one thing', 'another thing']
>>> print "Changed " + ", ".join(map(str, updated_list[:-1])) + " and " + updated_list[-1] + "."
Changed one thing and another thing.
>>> updated_list = ['this thing', 'that thing', 'that other thing']
>>> print "Changed " + ", ".join(map(str, updated_list[:-1])) + " and " + updated_list[-1] + "."
Changed this thing, that thing and that other thing.

Feedback Welcome

About Me, Python, Technical

Why Python Can Suck (But I Still Like It)

Link: http://dannyman.toldme.com/2009/12/08/perl-vs-python-documentation/

Based on something I posted to Facebook:

Perl’s “natural language” emphasis and non-obvious locutions encourage developers to document their code. Many times I have put some hard work into a line or a block of code, and ended up writing in some comments as a part of the process.

Python code is nice and readable but often very poorly documented. “The code should speak for itself,” and while it is easier to read Python code sometimes a few human-language comments would save some time and annoyance.

Maintaining Python code is easier than maintaining Perl, but using Perl modules is often easier than using Python modules because Perl developers are kinda forced to explain their work.

One of the first things I had to accept about pydoc is it is almost universally worthless, and even good functional documentation will often omit the sort of useful examples that I am most likely to find quickly intuitive.

This is why I have transitioned so slowly. Python’s strengths breed a certain weakness just as Perl’s weaknesses breed a certain strength.

At this point, I would consider myself approaching bi-linguality, and pretty comfortable in either language. If I have a preference it would be for Python, because I am lazy about documentation and even maintaining my own code, it is a lot easier to figure out what I wrote in Python somewhat after the fact. But my fondness for Perl and its generally more approachable documentation stands, and I’m not about to dis what has served me so long and so well.

1 Comment

Python, Technical

Notes on Porting a Django App from SQLite to MySQL

Link: http://dannyman.toldme.com/2009/12/04/django-migrate-database-sqlite-mysql/

This was painful. I’ll be doing this again next week, then hopefully never again.

First off, if you ever hit an error, drop and re-create your MySQL database, or at least drop the tables, or things just get weirder.

Grab the data from SQLite:

python manage.py dumpdata > $APPNAME.json

The, update settings.py to connect to the MySQL database, and if you are really lucky:

python manage.py syncdb
python manage.py reset contenttypes
python manage.py loaddata $APPNAME

(Thank you, Carl Meyer!)

Now, here’s the cute things I ran into. I had originally built a model with a TextField primary key. This is fine by SQLite but when Django tries to create a BLOB field you get in trouble asking for it to be unique, never mind a primary key. Fortunately, it was easy enough for me to simply change it to a CharField, which will tell Django to use VARCHAR. (SQLite certainly didn’t mind.)

The other was that neither Django nor SQLite were enforcing field length limitations, so I would hit some errors when loaddata tried to bring in database entries that were too long. I worked around this by raising my length limitations. There was also some ugliness with a UTF-8 string, which I solved by creating the text object in question.


Python, Technical

HOWTO: Parse Recent Twiki Edits in Python

Link: http://dannyman.toldme.com/2009/06/11/howto-python-feedparser-twiki-rss/

To make up for my snarkiness in my last post . . . it is an easy matter of fetching the WebRss node from Twiki and running it through the Universal Feed Parser:

# Twiki RSS Feed
import feedparser
import time
import calendar
# http://www.feedparser.org/
d = feedparser.parse(twiki_rss_url)
for e in d['entries']:
# e.updated_parsed = tuple UTC
# calendar.timegm = seconds UTC
# time.localtime = tuple locale
print time.strftime("%Y-%m-%d %H:%M",
print e.rdf_value # Author
print e.title

1 Comment

Site Archive