Shotgun Jira Bridge Issue

Pipeline Integrations
,
1

Hi all,
Hope you all are well today! :slight_smile:

I am trying to set up the Shotgun-Jira bridge. I’ve followed the documentation thoroughly. I have my own Jira server instance on-premise. When I launch this command line,

python /home/geoff/workspace/shotgunJira/sg-J=jira-bridge/webapp.py --settings /home/geoff/workspace/shotgunJira/sg-jira-bridge/settings.py --port 9090

I have a Traceback like this:

2019-09-17 21:48:52,275 INFO [bridge] Successfully read settings from /home/geoff/workspace/shotgunJira/sg-jira-bridge/settings.py
2019-09-17 21:48:52,969 INFO [shotgun_session] Connected to https://my-awesome-website.shotgunstudio.com.
2019-09-17 21:48:54,577 INFO [jira_session] Connected to http://my-awesome-jira-site.com
2019-09-17 21:48:55,215 ERROR [bridge] Missing required custom Shotgun Project field sg_jira_key
Traceback (most recent call last):
  File "/home/geoff/workspace/shotgunJira/sg-jira-bridge/sg_jira/bridge.py", line 146, in get_bridge
    sg_http_proxy=shotgun_settings.get("http_proxy"),
  File "/home/geoff/workspace/shotgunJira/sg-jira-bridge/sg_jira/bridge.py", line 86, in __init__
    self._shotgun.setup()
  File "/home/geoff/workspace/shotgunJira/sg-jira-bridge/sg_jira/shotgun_session.py", line 96, in setup
    "text"
  File "/home/geoff/workspace/shotgunJira/sg-jira-bridge/sg_jira/shotgun_session.py", line 114, in assert_field
    entity_type, field_name,
RuntimeError: Missing required custom Shotgun Project field sg_jira_key
Traceback (most recent call last):
  File "/home/geoff/workspace/shotgunJira/sg-jira-bridge/webapp.py", line 332, in <module>
    main()
  File "/home/geoff/workspace/shotgunJira/sg-jira-bridge/webapp.py", line 326, in main
    certfile=certfile,
  File "/home/geoff/workspace/shotgunJira/sg-jira-bridge/webapp.py", line 278, in run_server
    ("localhost", port), RequestHandler
  File "/home/geoff/workspace/shotgunJira/sg-jira-bridge/webapp.py", line 56, in __init__
    self._sg_jira = sg_jira.Bridge.get_bridge(settings)
  File "/home/geoff/workspace/shotgunJira/sg-jira-bridge/sg_jira/bridge.py", line 146, in get_bridge
    sg_http_proxy=shotgun_settings.get("http_proxy"),
  File "/home/geoff/workspace/shotgunJira/sg-jira-bridge/sg_jira/bridge.py", line 86, in __init__
    self._shotgun.setup()
  File "/home/geoff/workspace/shotgunJira/sg-jira-bridge/sg_jira/shotgun_session.py", line 96, in setup
    "text"
  File "/home/geoff/workspace/shotgunJira/sg-jira-bridge/sg_jira/shotgun_session.py", line 114, in assert_field
    entity_type, field_name,
RuntimeError: Missing required custom Shotgun Project field sg_jira_key

Here are my settings.py:
settings.py (3.9 KB)

and my .env file (sorry for the .py extension. It was to allow the upload of the file):
dot_env.py (551 Bytes)

I’ve changed the websites addresses for security reasons.

The thing I don’t understand in the traceback is that the script complains about a missing field

RuntimeError: Missing required custom Shotgun Project field sg_jira_key

On the picture below, I believe that I’ve added the correct custom field

shotgunjira_01
shotgunjira_01.png1199×280 41.2 KB

and the jira project is Shotgun-jira:

shotgunjira_02

Does anyone have an idea about what is causing the issue?
Thanks a lot for your help! :slight_smile:

Geoff

5 Likes
2

Hi Geoff,

I noticed in your screenshot that the column header reads sg_jira_key. When you created the column, did you call it sg_jira_key or jira_key?

I ask this because Shotgun prepends the sg automatically thus calling the field jira_key would yield a column with a field code of sg_jira_key and calling it sg_jira_key would yield a field code of sg_sg_jira_key.

You can verify this by configuring the field (right click on header -> Configure Field) and looking at the field code to the right of the name in the dialog box that pops up.

9 Likes
3

Hi Patrick,
Many thanks for your email.

You are absolutely right! this is the issue.
I’ve been confused about the display name and the actual field name!

When I launch the bridge web server, it doesn’t complains anymore (yay!)
but I have some issues with the shotgunEvents that give me some errors.
I’ll investigate first before bothering you!

Again thank you so much for your help, much appreciated! :slight_smile:
Geoff

6 Likes
5

Okay! So I’m getting closer to make it work! :slight_smile:
The shotgunEventDaemon.py is complaining about a connection:

cd /usr/local/shotgun/shotgunEvents/src
export SGDAEMON_SGJIRA_NAME="sg_jira_event_trigger"
export SGDAEMON_SGJIRA_KEY="myAwes0meAP1Key"
source /home/geoff/workspace/shotgunJira/shotgunJiraVenv/bin/activate 
./shotgunEventDaemon.py foreground

Traceback:

INFO:engine:Using Shotgun version 3.1.1
INFO:engine:Loading plugin at /home/geoff/workspace/shotgunJira/sg-jira-bridge/triggers/__init__.py
Traceback (most recent call last):
  File "./shotgunEventDaemon.py", line 1083, in emit
    smtp.connect(self.mailhost, port)
  File "/usr/local/lib/python2.7/smtplib.py", line 317, in connect
    self.sock = self._get_socket(host, port, self.timeout)
  File "/usr/local/lib/python2.7/smtplib.py", line 292, in _get_socket
    return socket.create_connection((host, port), timeout)
  File "/usr/local/lib/python2.7/socket.py", line 575, in create_connection
    raise err
error: [Errno 111] Connection refused
Logged from file shotgunEventDaemon.py, line 776
CRITICAL:engine:Did not find a registerCallbacks function in plugin at /home/geoff/workspace/shotgunJira/sg-jira-bridge/triggers/__init__.py.
INFO:engine:Loading plugin at /home/geoff/workspace/shotgunJira/sg-jira-bridge/triggers/sg_jira_event_trigger.py
DEBUG:engine:Starting the event processing loop.

So if I follow the documentation
By creating an asset or a task in shotgun and I have several tracebacks that comes with it.

Here are the logs that have recorded the entire process:

plugin.init.log|attachment (5.8 KB)
plugin.sg_jira_event_trigger.log (334.1 KB)
shotgunEventDaemon.log (2.7 KB)

I have added the extension .log to be able to uplaod them in the forum, but they are coming without extension obviously :slight_smile:

Again thank you so much for everything, I really appreciate your help!
Geoff.

5 Likes
6

The traceback you quoted mentions smtplib.py. Out of curiosity, did you configure the email notification section of the event daemon’s config file?

2 Likes
7

The only thing I’ve changed was the email address from and to:

[emails]
# Email notification settings. These are used for error reporting because we
# figured you wouldn't constantly be tailing the log and would rather have an
# active notification system.
#
# Any error above level 40 (ERROR) will be reported via email.
#
# All of these value must be provided for there to be email alerts sent out.

# The server that should be used for smtp connections. The username and password
# values can be uncommented to supply credentials for the smtp connection. The
# smtp port can also be uncommented to supply an alternate port such as 587 for
# GMail TLS SMTP. As for the useTLS parameter, it is a True/False value.
server: smtp.yourdomain.com
#port: 587
#useTLS: True
#username: yourSMTPUsername
#password: yourSMPTPassword

# The from address that should be used in emails.
from: geoff@gmail.com

# A comma delimited list of email addresses to whom these alerts should be sent.
to: geoff@gmail.com

# An email subject prefix that can be used by mail clients to help sort out
# alerts sent by the Shotgun event framework.
subject: [SG]

Should I restore the default values?

1 Like
8

btw I’ve just realized that the email address is not right (Don’t ask me why!).
could it be the issue?

1 Like
9

You’ll need to setup the server as well and the username and password if the server requires it. That should take care of the smtp server connection error.

2 Likes
10

sure doing it right now and I’ll let you know the results of it.
Thanks Patrick!

2 Likes
11

So here is the email configuration part of shotgunEventDaemon.conf:

# Email notification settings. These are used for error reporting because we
# figured you wouldn't constantly be tailing the log and would rather have an
# active notification system.
#
# Any error above level 40 (ERROR) will be reported via email.
#
# All of these value must be provided for there to be email alerts sent out.

# The server that should be used for smtp connections. The username and password
# values can be uncommented to supply credentials for the smtp connection. The
# smtp port can also be uncommented to supply an alternate port such as 587 for
# GMail TLS SMTP. As for the useTLS parameter, it is a True/False value.
server: smtp.gmail.com
port: 587
useTLS: True
username: MyGmailUserName
password: My_Super_secret_password

# The from address that should be used in emails.
from: MyGmailUserName@gmail.com

# A comma delimited list of email addresses to whom these alerts should be sent.
to: MyGmailUserName@gmail.com

# An email subject prefix that can be used by mail clients to help sort out
# alerts sent by the Shotgun event framework.
subject: [SG]

And I’ve got a traceback when I run ./shotgunEventDaemon.py foreground

./shotgunEventDaemon.py foreground

INFO:engine:Using Shotgun version 3.1.1
INFO:engine:Loading plugin at /home/geoff/workspace/shotgunJira/sg-jira-bridge/triggers/__init__.py
Traceback (most recent call last):
  File "./shotgunEventDaemon.py", line 1095, in emit
    smtp.login(self.username, self.password)
  File "/usr/local/lib/python2.7/smtplib.py", line 623, in login
    raise SMTPAuthenticationError(code, resp)
SMTPAuthenticationError: (534, '5.7.9 Application-specific password required. Learn more at\n5.7.9  https://support.google.com/mail/?p=InvalidSecondFactor f197sm6925774wme.22 - gsmtp')
Logged from file shotgunEventDaemon.py, line 776
CRITICAL:engine:Did not find a registerCallbacks function in plugin at /home/geoff/workspace/shotgunJira/sg-jira-bridge/triggers/__init__.py.
INFO:engine:Loading plugin at /home/geoff/workspace/shotgunJira/sg-jira-bridge/triggers/sg_jira_event_trigger.py
DEBUG:engine:Starting the event processing loop.
DEBUG:engine:Got 1 events: 467116 to 467116.
DEBUG:plugin.__init__:Skipping: inactive.

On this account I have a 2 step verification system I would guess the error comes from this isn’t it?

2 Likes
12

Hmmm… Yeah, probably 2 step in the way here. Might want to disable it just for a test.

I think some systems have app specific password that you can generate to supply to systems that don’t support more complex authentication mechanisms. Looks like Google offers this option:

https://support.google.com/accounts/answer/185833?hl=en

4 Likes
13

I’ll have a look at this today and let you know the result of this.
As usual, Thank you so much Patrick for your precious help.

Geoff

4 Likes
14

Hi all!

So Patrick, your suggestion helped a lot. Now thanks to this, the shotgunEventDaemon doesn’t complain anymore about authentification (hurray! Another step forward! :smiley: )!

So I’ve created an asset in shotgun with a task template VFX. From there I’ve created a new task (comp) and I’ve ticked the “Sync in Jira” checkbox.

I attached the new log that shows in line 69, 158, 232, 318, 392 and onward the same traceback:

ConnectionError: HTTPConnectionPool(host='myAwesomeSite', port=9090): Max retries exceeded with url: /sg2jira/default/Asset/1406 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f1f6421fcd0>: Failed to establish a new connection: [Errno 111] Connection refused',))

plugin.sg_jira_envent_trigger.log (363.2 KB)

The webapp doesn’t complain at all.

I have the feeling we are nearly there!
Thanks all for your help! :slight_smile:

Geoff.

3 Likes
15

Hey Geoff,

:thinking:

Is http://myAwesomeSite:9090 (your Jira bridge server’s URL) accessible from the system running the event daemon? Might there be a firewall, proxy or something else in the way hampering communications between the daemon and the sync server?

@kporangehat any thoughts here? I’m afraid I might be drawing a blank.

The last interesting line before the first traceback is:

DEBUG:plugin.sg_jira_event_trigger.process_event:Posting event {'entity_id': 1406, 'attribute_name': 'code', 'in_create': True, 'entity_type': 'Asset', 'old_value': None, 'new_value': 'weapon', 'field_data_type': 'text', 'type': 'attribute_change'} to http://myAwesomeSite:9090/sg2jira/default/Asset/1406

which looks fine to me (unless the site is unreachable).

You could try running the following from interactive Python interpreter on the machine that runs the event daemon to validate independently if it is an infrastructure issue or an app / config issue:

import socket

def has_connection(hostname, port):
    try:
        host = socket.gethostbyname(hostname)
        socket.create_connection((host, port), 2)
        return True
    except socket.gaierror:
        pass
    return False

has_connection('myAwesomeSite', 9090)
2 Likes
16

Hey Patrick,
I have a very small setup for dev purpose, but basically I am running an Ubuntu Server that have Jira and the event daemon on the same machine (as well as the webapp).
Would this be an issue?

All are accessible from the outside world. I’ll PM you the address if you want.
Please let me know! :slight_smile:

Thanks!
Geoff

1 Like
17

Ok. So…
After having tried running the script, it is returning False …

My next move will be to probably create a cloud server or lambda AWS function for the eventDaemon or the web app.

It is weird because if I type:
sudo lsof -i -P -n | grep LISTEN
I can see the webapp running (the last line):

systemd-r   666 systemd-resolve   13u  IPv4   20636      0t0  TCP 127.0.0.53:53 (LISTEN)
cupsd       882            root    6u  IPv6   27891      0t0  TCP [::1]:631 (LISTEN)
cupsd       882            root    7u  IPv4   27892      0t0  TCP 127.0.0.1:631 (LISTEN)
mongod     1225         mongodb   11u  IPv4   31005      0t0  TCP 127.0.0.1:27017 (LISTEN)
container  1229            root    7u  IPv4   28573      0t0  TCP 127.0.0.1:39557 (LISTEN)
sshd       1241            root    3u  IPv4   30715      0t0  TCP *:22 (LISTEN)
sshd       1241            root    4u  IPv6   30717      0t0  TCP *:22 (LISTEN)
xrdp-sesm  1310            root    7u  IPv6   26123      0t0  TCP [::1]:3350 (LISTEN)
xrdp       1377            xrdp   11u  IPv6   28043      0t0  TCP *:3389 (LISTEN)
postgres   1775        postgres    5u  IPv4   29180      0t0  TCP 127.0.0.1:5432 (LISTEN)
mysqld    12476           mysql   31u  IPv4  127946      0t0  TCP 127.0.0.1:3306 (LISTEN)
java      12766            root  284u  IPv6  133856      0t0  TCP *:8080 (LISTEN)
java      12766            root  296u  IPv6  131998      0t0  TCP 127.0.0.1:8005 (LISTEN)
python    30888           geoff    3u  IPv4 5453092      0t0  TCP 127.0.0.1:9090 (LISTEN)

Or is it a firewall issue?
Should I try using 127.0.0.1 or 0.0.0.0 instead of the web address in the conf file?

No matter what, let’s keep working on it! :slight_smile:

1 Like
18

Hi Geoff,

I’d definitely try with 127.0.0.1:9090 if you haven’t already. Also checking any firewall does sound like a good idea. This definitely seems network related at this point.

5 Likes
19

Hi Patrick,
I’ll let you know how it went.
You’ve been an absolute legend with me. Thank you so much.

Speak to you soon! :wink:

5 Likes
20

A couple of additional things:

  • Are you getting any warnings or errors when starting up the jira bridge webapp server?
  • Can you hit http://myAwesomeSite:9090 from a web browser? (make sure you’re using the right port number I saw 9009 and 9090 mentioned in different comments in this thread, but 9090 is the default.
  • myAwesomeSite may not be resolving correctly. I’d recommend using localhost or 127.0.0.1 to remove that from the possibilities.
  • The firewall idea from @bouchep is another good place to look

cheers,
kp

6 Likes
21

9009 was a typo on my part. My bad. :grimacing:

I edited my reply to reflect the right port.

Thanks KP!

3 Likes