staging.inyokaproject.org

Hallo Zusammen..

ich stehe vor einer herausforderung, die wenn man Ahnung hat warscheinlich keine ist. da ich jedoch keine Habe ☺ ist es für mich eine...

ich erhalte infos im JSON format und muss alles oder besser Teile in eine anderes Format bringen.

ich habe das Program JQ gefunden das eigentlich wie gemacht dafpr ist, jedoch mit der Komplexität der struktur probleme zu habven scheint? oder ich raff es nicht.

hier mal die Quelle:

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158

{
    "request": {
        "subject": "PROJEKT: Umzug VMLVS Server",
        "resolution": {
            "resolution_attachments": [],
            "content": null
        },
        "linked_to_request": null,
        "mode": {
            "name": "E-Mail",
            "id": "1"
        },
        "lifecycle": null,
        "assets": [],
        "is_trashed": false,
        "id": "67276",
        "assigned_time": {
            "display_value": "May 21, 2019 10:29 AM",
            "value": "1558427380668"
        },
        "group": {
            "site": {
                "id": 602
            },
            "name": "System-/Netzwerkadministration",
            "id": "603"
        },
        "requester": {
            "email_id": "user.user@uder.de",
            "name": "user, user",
            "is_vipuser": false,
            "id": "118802",
            "department": {
                "site": {
                    "name": "Verwaltung",
                    "id": 602
                },
                "name": "IT",
                "id": 6902
            }
        },
        "email_to": [],
        "created_time": {
            "display_value": "May 21, 2019 10:28 AM",
            "value": "1558427318375"
        },
        "item": null,
        "level": {
            "name": "Anfrage",
            "id": "1"
        },
        "has_resolution_attachments": false,
        "approval_status": null,
        "sla": {
            "name": "(***) Anfrage",
            "id": "610"
        },
        "priority": {
            "color": "#0099ff",
            "name": "***",
            "id": "2"
        },
        "created_by": {
            "email_id": "user.user@user.de",
            "name": "user, user",
            "is_vipuser": false,
            "id": "118802",
            "department": {
                "site": {
                    "name": "Verwaltung",
                    "id": 602
                },
                "name": "IT",
                "id": 6902
            }
        },
        "first_response_due_by_time": null,
        "last_updated_time": {
            "display_value": "Jan 28, 2020 12:45 PM",
            "value": "1580211911222"
        },
        "has_notes": true,
        "udf_fields": {
            "udf_pick_1507": null,
            "udf_pick_1506": "nein"
        },
        "subcategory": null,
        "email_cc": [],
        "status": {
            "color": "#0066ff",
            "name": "Open",
            "id": "1"
        },
        "template": {
            "name": "Default Request",
            "id": "601"
        },
        "email_ids_to_notify": [],
        "description": "<div>VMLVS08-1&nbsp; --&gt; DELVS1<br /></div><div>VMLVS08-2&nbsp; --&gt; DELVS2<br /></div><div>VMLVS08-ESP&nbsp; --&gt; DELVSES<br /></div><div><br /></div><div>VMFUNK08&nbsp; --&gt; DEFUNK (Feste IP)<br /></div>",
        "has_dependency": false,
        "has_conversation": true,
        "callback_url": null,
        "is_service_request": false,
        "urgency": {
            "name": "Normal",
            "id": "3"
        },
        "is_shared": false,
        "has_request_initiated_change": false,
        "request_template_task_ids": [],
        "department": {
            "site": {
                "id": "602"
            },
            "name": "IT",
            "id": "6902"
        },
        "is_reopened": false,
        "has_draft": false,
        "has_attachments": false,
        "has_linked_requests": false,
        "is_overdue": true,
        "technician": {
            "email_id": "user.user@user.de",
            "name": "user, user",
            "is_vipuser": false,
            "id": "118802",
            "department": {
                "site": {
                    "name": "Verwaltung",
                    "id": 602
                },
                "name": "IT",
                "id": 6902
            }
        },
        "has_request_caused_by_change": false,
        "has_problem": false,
        "due_by_time": {
            "display_value": "May 30, 2019 10:28 AM",
            "value": "1559204918375"
        },
        "has_project": false,
        "site": {
            "name": "Verwaltung",
            "id": "602"
        },
        "is_first_response_overdue": false,
        "category": null
    },
    "response_status": {
        "status_code": 2000,
        "status": "success"
    }
}

wünschenwerd wäre nun das so zu transformieren das ich einzelne inhalte in eine Flache strucktur bringen kann:

1
2
3
4
5
6
7
8

{
request.subjesubject:"PROJEKT: Umzug VMLVS Server",
request.id:"67276",
request.requester.email_id:"user.user@uder.de",
request.requester.requester.email_id:"user, user",
request.created_time.display_value:"May 21, 2019 10:28 AM",
}

geht das mit JQ überhaupt? oder nehme ich besser was anderes?

Der Syntaxhighlighter lässt Zweifel daran aufkommen, dass das gewünschte Forma JSON ist. Hast Du da die Quotes vergessen?

Ja, tatsächlich...

Mit Python3 könnte man z.B. sowas machen, wenn man das json glätten und filtern will:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

#!/usr/bin/env python3
from collections.abc import MutableMapping
import json
import sys

# add all keys you want to use to the following list:
keys = [
    'request.subject',
    'request.id',
    'request.requester.email_id',
    'request.requester.name',
    'request.created_time.display_value',
]


def flatten(d, parent_key='', sep='_'):
    items = []
    for k, v in d.items():
        new_key = parent_key + sep + k if parent_key else k
        if isinstance(v, MutableMapping):
            items.extend(flatten(v, new_key, sep=sep).items())
        else:
            items.append((new_key, v))
    return dict(items)


for p in sys.argv[1:]:
    print('#' * 10, p, '#' * 10)
    with open(p) as f:
        content = json.load(f)
    flattened = flatten(content, sep='.')
    result = {k: flattened[k] for k in keys}
    print(json.dumps(result, sort_keys=True, indent=4, separators=(',', ': ')))

Das Skript akzeptiert eine oder mehrere JSON-Dateien als Argumente.

Statt JSON kann man natürlich z.B. auch einfach nur die Keys und Values des Dictionary ausgeben lassen.

vielen Dank, probiere es mal...

Du musst python3 nehmen, python zeigt auf python2 und das hat kein abstract base class (abc) Modul in den collections - das kam erst mit Python 3.3: https://docs.python.org/3/library/collections.abc.html

Vielen Dank, das geht echt gut.. das mit Phyton 3 hatte ich schon gemerkt, desswegen habe ich die antwort schon geändert...

kleine Frage habe ich allerdings noch. kann ich nachträglich noch ein feld einfügen z.b.

ID: "1234567"

also einfach zur flachen strucktur...

mit jq sollte das eigentlich gehen...

 jq test.json + "id":"1234567";

tut es aber nicht ☺

wobei die Syntax wohl lauten sollte :

cat test.json | jq "id":"777";+

bekomme dann aber:

1
2
3
4

jq: error: syntax error, unexpected ':', expecting $end (Unix shell quoting issues?) at <top-level>, line 1:
id:777
jq: 1 compile error
+: command not found

KingArtus schrieb:

kann ich nachträglich noch ein feld einfügen z.b.

ID: "1234567"

also einfach zur flachen strucktur...

Ja, einfach das result Dictionary erweitern - man könnte das Skript z.B. so umgestalten, um beliebige JSON-Objekte in das Ergebnis einfügen und bei Bedarf zusätzlich Keys angeben zu können, die aus den geglätteten JSON-Objekten selektioniert werden sollen:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51

#!/usr/bin/env python3
from argparse import ArgumentParser
from collections.abc import MutableMapping
import json
import sys

parser = ArgumentParser(description="flatten json object and add additional values")
parser.add_argument("json", metavar="JSON", nargs="+", help="json file(s) to process")
parser.add_argument("-a", "--add-json", metavar="ADD_JSON", action="append",
                    default=[], help="add json objects to output")
parser.add_argument("-k", "--filter-key", metavar="FILTER_KEY", action="append",
                    default=[], help="add keys to select from the flattened result")
args = parser.parse_args()

# add all keys you want to use to the following list:
keys = [
    'request.subject',
    'request.id',
    'request.requester.email_id',
    'request.requester.name',
    'request.created_time.display_value',
]
keys.extend(args.filter_key)


def flatten(d, parent_key='', sep='_'):
    items = []
    for k, v in d.items():
        new_key = parent_key + sep + k if parent_key else k
        if isinstance(v, MutableMapping):
            items.extend(flatten(v, new_key, sep=sep).items())
        else:
            items.append((new_key, v))
    return dict(items)

addition_dict = {}

for addition in args.add_json:
    try:
        addition_dict.update(json.loads(addition))
    except json.JSONDecodeError:
       print("invalid json:", addition, file=sys.stderr)

for p in args.json:
    print('#' * 10, p, '#' * 10)
    with open(p) as f:
        content = json.load(f)
    flattened = flatten(content, sep='.')
    result = {k: flattened.get(k) for k in keys}
    result.update(addition_dict)
    print(json.dumps(result, sort_keys=True, indent=4, separators=(',', ': ')))

Für das Beispiel mit der ID sähe das z.B. so aus:

./flatten_json.py input.json -a '{"id": "1234567"}' 

Vielen dank, teste ich gleich mal aus... habe dein Script so angepasst das es auch geht allerdings nicht mehr nachträglich...

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44

#!/usr/bin/env python3
from collections.abc import MutableMapping
import json
import sys

# add all keys you want to use to the following list:
keys = [
    'request.subject',
    'request.id',
    'request.requester.email_id',
    'request.requester.name',
    'request.created_time.display_value',
    'request.group.name',
    'request.created_by.email_id',
    'request.created_by.name',
    'request.created_by.department.site.name',
    'request.created_by.department.name',
    'request.description',
    'request.technician.email_id',
    'request.technician.name',
    'request.status.name',
]


def flatten(d, parent_key='', sep='_'):
    items = []
    for k, v in d.items():
        new_key = parent_key + sep + k if parent_key else k
        if isinstance(v, MutableMapping):
            items.extend(flatten(v, new_key, sep=sep).items())
        else:
            items.append((new_key, v))
    return dict(items)


for p in sys.argv[3:]:
    print('#' * 10, p, '#' * 10)
    with open(p) as f:
        content = json.load(f)
    flattened = flatten(content, sep='.')
    result = {k: flattened[k] for k in keys}
    result[sys.argv[1]] = sys.argv[2]
    print(json.dumps(result, sort_keys=True, indent=4, separators=(',', ': ')))

das geht aber irgendwie erhalte ich manchmal fehler, und manchmal nicht.. ich lasse es in einer schleife laufen.

hier der fehler:

1
2
3
4
5
6
7

root@delxsearch:/docs/transform/in# python3 /docs/script/transform.py id 70840 /docs/transform/in/70840.json > /docs/transform/out/70840.json
Traceback (most recent call last):
  File "/docs/script/transform.py", line 41, in <module>
    result = {k: flattened[k] for k in keys}
  File "/docs/script/transform.py", line 41, in <dictcomp>
    result = {k: flattened[k] for k in keys}
KeyError: 'request.created_by.department.site.name'

hier noch die Quelldatei:

root@delxsearch:/docs/transform/in# cat 70840.json
{"request":{"subject":"EDI-Auftragsübernahme: Zukünftiger real Zentralauftrag - Artikel in Display umwandeln","resolution":{"resolution_attachments":[],"content":null},"linked_to_request":null,"mode":{"name":"E-Mail","id":"1"},"lifecycle":null,"assets":[],"is_trashed":false,"id":"70840","assigned_time":{"display_value":"Jan 2, 2020 01:23 PM","value":"1577967792002"},"group":{"site":{"id":602},"name":"Anwendungsentwicklung","id":"604"},"requester":{"email_id":"keyuservertrieb@pjdgj.de","name":"Keyuservertrieb","is_vipuser":false,"id":"130187","department":{"site":{"name":"Verwaltung Kall","id":602},"name":"Sales","id":4209}},"email_to":["helpdesk@püopr.de"],"created_time":{"display_value":"Jan 2, 2020 11:41 AM","value":"1577961712486"},"item":{"name":"ORDERS","id":"1505"},"level":{"name":"Anfrage","id":"1"},"has_resolution_attachments":false,"approval_status":null,"sla":{"name":"(***) Anfrage","id":"610"},"resolved_time":{"display_value":"Jan 10, 2020 09:06 AM","value":"1578643610287"},"priority":{"color":"#0099ff","name":"***","id":"2"},"created_by":{"email_id":null,"name":"System","is_vipuser":false,"id":"1","department":null},"first_response_due_by_time":null,"last_updated_time":{"display_value":"Feb 3, 2020 09:36 AM","value":"1580718985372"},"has_notes":false,"udf_fields":{"udf_pick_1507":"Neuanlage / Konfiguration","udf_pick_1506":"nein"},"subcategory":{"name":"EDI","id":"901"},"email_cc":["meladgfjde.redgjr@pjgjgjghjar.de"],"status":{"color":"#006600","name":"Closed","id":"3"},"template":{"name":"Default Request","id":"601"},"email_ids_to_notify":[],"time_elapsed":{"display_value":"3hrs 43min","value":"13427602"},"description":"<div><p class=\"MsoNormal\"><span class=\"colour\" style=\"color: rgb(31, 73, 125)\">Hallo und ein frohes neues Jahr,<\/span><br /><\/p><p class=\"MsoNormal\"><span class=\"colour\" style=\"color: rgb(31, 73, 125)\">&nbsp;<\/span><br /><\/p><p class=\"MsoNormal\"><span class=\"colour\" style=\"color: rgb(31, 73, 125)\">anbei eine Anfrage von Melanie Reger, mit der Bitte um Prüfung auf Umsetzbarkeit.<\/span><br /><\/p><p class=\"MsoNormal\"><span class=\"colour\" style=\"color: rgb(31, 73, 125)\">&nbsp;<\/span><br /><\/p><p class=\"MsoNormal\"><span class=\"colour\" style=\"color: rgb(31, 73, 125)\">Es handelt sich um 206 Häuser, die insgesamt 241 Displays bestellen werden.<\/span><br /><\/p><p class=\"MsoNormal\"><span class=\"colour\" style=\"color: rgb(31, 73, 125)\">Die Zentralaufträge kommen via EDI rein.<\/span><br /><\/p><p class=\"MsoNormal\"><span class=\"colour\" style=\"color: rgb(31, 73, 125)\">&nbsp;<\/span><br /><\/p><p class=\"MsoNormal\"><span class=\"colour\" style=\"color: rgb(31, 73, 125)\">Vielen Dank.<\/span><br /><\/p><p class=\"MsoNormal\"><span class=\"colour\" style=\"color: rgb(31, 73, 125)\">&nbsp;<\/span><br /><\/p><p class=\"MsoNormal\"><span class=\"colour\" style=\"color: rgb(31, 73, 125)\">&nbsp;<\/span><br /><\/p><p class=\"MsoNormal\"><span class=\"colour\" style=\"color: rgb(31, 73, 125)\">Gruß<\/span><br /><\/p><p class=\"MsoNormal\"><span class=\"colour\" style=\"color: rgb(31, 73, 125)\">&nbsp;<\/span><br /><\/p><div><table style=\"width: 435pt\" width=\"0\" cellspacing=\"0\" cellpadding=\"0\" border=\"0\"><tbody><tr style=\"height: 78pt\"><td style=\"width: 91.5pt; padding: 0cm 0cm 0cm 0cm; height: 78pt\" width=\"122\"><p class=\"MsoNormal\"><span class=\"colour\" style=\"color: rgb(31, 73, 125)\"><span class=\"font\" style=\"font-family: &quot;Times New Roman&quot;, serif\"><span class=\"size\" style=\"font-size: 12pt\"><img alt=\"PAPSTAR Logo\" src=\"/inlineimages/WorkOrder/70840/image001.jpg\" style=\"width: 0.927in; height: 1.0416in\" width=\"0.927in\" height=\"1.0416in\" /><\/span><\/span><\/span><br /><\/p><\/td><td style=\"width: 124.5pt; padding: 0cm 0cm 0cm 0cm; height: 78pt\" width=\"164\" valign=\"top\"><p class=\"MsoNormal\"><span class=\"colour\" style=\"color: rgb(31, 73, 125)\"><span class=\"font\" style=\"font-family: &quot;Times New Roman&quot;, serif\"><span class=\"size\" style=\"font-size: 7.5pt\">&nbsp;<\/span><span class=\"size\" style=\"font-size: 12pt\"><\/span><\/span><\/span><br /><\/p><table cellspacing=\"0\" cellpadding=\"0\" border=\"0\"><tbody><tr><td style=\"padding: 0cm 0cm 0cm 0cm\"><p class=\"MsoNormal\"><b><span class=\"colour\" style=\"color: black\"><span class=\"font\" style=\"font-family: &quot;Arial&quot;, sans-serif\"><span class=\"size\" style=\"font-size: 10pt\">PAPSTAR GmbH<\/span><\/span><\/span><\/b><span class=\"colour\" style=\"color: black\"><span class=\"font\" style=\"font-family: &quot;Times New Roman&quot;, serif\"><span class=\"size\" style=\"font-size: 10pt\"><\/span><\/span><\/span><br /><\/p><\/td><\/tr><tr><td style=\"padding: 0cm 0cm 0cm 0cm\"><p class=\"MsoNormal\"><span class=\"colour\" style=\"color: black\"><span class=\"font\" style=\"font-family: &quot;Arial&quot;, sans-serif\"><span class=\"size\" style=\"font-size: 10pt\">Daimlerstraße 4-8<\/span><\/span><span class=\"font\" style=\"font-family: &quot;Times New Roman&quot;, serif\"><span class=\"size\" style=\"font-size: 10pt\"><\/span><\/span><\/span><br /><\/p><\/td><\/tr><tr><td style=\"padding: 0cm 0cm 0cm 0cm\"><p class=\"MsoNormal\"><span class=\"colour\" style=\"color: black\"><span class=\"font\" style=\"font-family: &quot;Arial&quot;, sans-serif\"><span class=\"size\" style=\"font-size: 10pt\">53925 Kall<\/span><\/span><span class=\"font\" style=\"font-family: &quot;Times New Roman&quot;, serif\"><span class=\"size\" style=\"font-size: 10pt\"><\/span><\/span><\/span><br /><\/p><\/td><\/tr><\/tbody><\/table><\/td><td style=\"width: 221.25pt; padding: 0cm 0cm 0cm 0cm; height: 78pt\" rowspan=\"3\" width=\"294\"><p class=\"MsoNormal\"><a href=\"https://www.papstar-storyteller.de/\"><span class=\"colour\" style=\"color: blue\"><span class=\"font\" style=\"font-family: &quot;Times New Roman&quot;, serif\"><span class=\"size\" style=\"font-size: 12pt\"><img alt=\"PAPSTAR Campaign\" src=\"/inlineimages/WorkOrder/70840/image002.jpg\" style=\"width: 3.0208in; height: 2.8125in\" width=\"3.0208in\" height=\"2.8125in\" border=\"0\" /><\/span><\/span><\/span><\/a><span class=\"colour\" style=\"color: rgb(31, 73, 125)\"><span class=\"font\" style=\"font-family: &quot;Times New Roman&quot;, serif\"><span class=\"size\" style=\"font-size: 12pt\"><\/span><\/span><\/span><br /><\/p><\/td><\/tr><tr style=\"height: 46.5pt\"><td style=\"padding: 0cm 0cm 0cm 0cm; height: 46.5pt\" colspan=\"2\" valign=\"top\"><table cellpadding=\"0\" border=\"0\"><tbody><tr><td style=\"padding: 0.75pt 0.75pt 0.75pt 0.75pt\" valign=\"top\"><p class=\"MsoNormal\"><span class=\"colour\" style=\"color: black\"><span class=\"font\" style=\"font-family: &quot;Arial&quot;, sans-serif\"><span class=\"size\" style=\"font-size: 10pt\">i. V. Dirk Vietoris<\/span><\/span><span class=\"font\" style=\"font-family: &quot;Times New Roman&quot;, serif\"><span class=\"size\" style=\"font-size: 10pt\"><\/span><\/span><\/span><br /><\/p><\/td><td style=\"border: none\" width=\"0\"><p class=\"MsoNormal\">&nbsp;<br /><\/p><\/td><\/tr><tr><td style=\"padding: 0.75pt 0.75pt 0.75pt 0.75pt\" rowspan=\"2\" valign=\"top\"><p class=\"MsoNormal\"><b><span class=\"colour\" style=\"color: black\"><span class=\"font\" style=\"font-family: &quot;Arial&quot;, sans-serif\"><span class=\"size\" style=\"font-size: 10pt\">Leiter Vertriebsinnendienst<\/span><\/span><\/span><\/b><span class=\"colour\" style=\"color: black\"><span class=\"font\" style=\"font-family: &quot;Times New Roman&quot;, serif\"><span class=\"size\" style=\"font-size: 10pt\"><\/span><\/span><\/span><br /><\/p><\/td><td style=\"border: none\" width=\"0\"><p class=\"MsoNormal\">&nbsp;<br /><\/p><\/td><\/tr><tr><td style=\"border: none\" width=\"0\"><p class=\"MsoNormal\">&nbsp;<br /><\/p><\/td><\/tr><\/tbody><\/table><\/td><\/tr><tr style=\"height: 75pt\"><td style=\"padding: 0cm 0cm 0cm 0cm; height: 75pt\" colspan=\"2\" valign=\"bottom\"><table style=\"width: 210pt\" width=\"0\" cellspacing=\"0\" cellpadding=\"0\" border=\"0\"><tbody><tr><td style=\"width: 45pt; padding: 0cm 0cm 0cm 0cm\" width=\"60\"><p class=\"MsoNormal\"><span class=\"colour\" style=\"color: black\"><span class=\"font\" style=\"font-family: &quot;Arial&quot;, sans-serif\"><span class=\"size\" style=\"font-size: 10pt\">Phone:<\/span><\/span><\/span><br /><\/p><\/td><td style=\"padding: 0cm 0cm 0cm 0cm\"><p class=\"MsoNormal\"><span class=\"colour\" style=\"color: black\"><span class=\"font\" style=\"font-family: &quot;Arial&quot;, sans-serif\"><span class=\"size\" style=\"font-size: 10pt\">+49 244183141<\/span><\/span><\/span><br /><\/p><\/td><\/tr><tr><td style=\"padding: 0cm 0cm 0cm 0cm\"><p class=\"MsoNormal\"><span class=\"colour\" style=\"color: black\"><span class=\"font\" style=\"font-family: &quot;Arial&quot;, sans-serif\"><span class=\"size\" style=\"font-size: 10pt\">Fax:<\/span><\/span><\/span><br /><\/p><\/td><td style=\"padding: 0cm 0cm 0cm 0cm\"><p class=\"MsoNormal\"><span class=\"colour\" style=\"color: black\"><span class=\"font\" style=\"font-family: &quot;Arial&quot;, sans-serif\"><span class=\"size\" style=\"font-size: 10pt\">+49 2441835141<\/span><\/span><\/span><br /><\/p><\/td><\/tr><tr><td style=\"padding: 0cm 0cm 0cm 0cm\"><p class=\"MsoNormal\"><span class=\"colour\" style=\"color: black\"><span class=\"font\" style=\"font-family: &quot;Arial&quot;, sans-serif\"><span class=\"size\" style=\"font-size: 10pt\">Email:<\/span><\/span><\/span><br /><\/p><\/td><td style=\"padding: 0cm 0cm 0cm 0cm\"><p class=\"MsoNormal\"><span class=\"colour\" style=\"color: black\"><span class=\"font\" style=\"font-family: &quot;Arial&quot;, sans-serif\"><span class=\"size\" style=\"font-size: 10pt\"><a title=\"Click to send email to Vietoris, Dirk\"><span class=\"colour\" style=\"color: black\">Döoo.Viioöris@paioööitar.de<\/span><\/a><\/span><\/span><\/span><br /><\/p><\/td><\/tr><tr><td style=\"padding: 0cm 0cm 0cm 0cm\"><p class=\"MsoNormal\"><span class=\"colour\" style=\"color: black\"><span class=\"font\" style=\"font-family: &quot;Arial&quot;, sans-serif\"><span class=\"size\" style=\"font-size: 10pt\">Web:<\/span><\/span><\/span><br /><\/p><\/td><td style=\"padding: 0cm 0cm 0cm 0cm\"><p class=\"MsoNormal\"><span class=\"colour\" style=\"color: black\"><span class=\"font\" style=\"font-family: &quot;Arial&quot;, sans-serif\"><span class=\"size\" style=\"font-size: 10pt\"><a href=\"http://www.papstar.com\"><span class=\"colour\" style=\"color: black\">www.papstar.com<\/span><\/a><\/span><\/span><\/span><br /><\/p><\/td><\/tr><\/tbody><\/table><\/td><\/tr><\/tbody><\/table><\/div><p class=\"MsoNormal\">&nbsp;<br /><\/p><div><div style=\"border: none; border-top: solid rgb(225, 225, 225) 1pt; padding: 3pt 0cm 0cm 0cm\"><p class=\"MsoNormal\"><b><span style=\"\">Von:<\/span><\/b><span style=\"\"> Reger, Melanie &lt;Meöiöie.Riöiöor@paiöiöir.de&gt; <br /> <b>Gesendet:<\/b> Donnerstag, 2. Januar 2020 11:01<br /> <b>An:<\/b> Keyuservertrieb &lt;keyuservertrieb@pä-jiä-i-itar.de&gt;<br /> <b>Betreff:<\/b> zukünftiger real Zentralauftrag Artikel in Display umwandeln<\/span><\/p><\/div><\/div><p class=\"MsoNormal\">&nbsp;<br /><\/p><p class=\"MsoNormal\">Hallo Dirk,<br /><\/p><p class=\"MsoNormal\">&nbsp;<br /><\/p><p class=\"MsoNormal\">real wird demnächst einen Zentralauftrag über das Display 66924 senden.<br /><\/p><p class=\"MsoNormal\">&nbsp;<br /><\/p><p class=\"MsoNormal\">Dieser wird jedoch wie folgt aussehen:<br /><\/p><p class=\"MsoNormal\"><span class=\"colour\" style=\"color: rgb(31, 73, 125)\">&nbsp;<\/span><br /><\/p><p class=\"MsoNormal\">EAN: <b>4002911370971<\/b><br /><\/p><p class=\"MsoNormal\">Lieferanten-ArtikelNr: 66924<br /><\/p><p class=\"MsoNormal\">Bezeichnung: Sammelnummer für Display 66924<br /><\/p><p class=\"MsoNormal\">Kartonmenge: 1<br /><\/p><p class=\"MsoNormal\">Bestellmenge: 241<br /><\/p><p class=\"MsoNormal\">&nbsp;<br /><\/p><p class=\"MsoNormal\">Die EAN ist die EAN des Artikels 88203.<br /><\/p><p class=\"MsoNormal\">Somit würde unser System ja erst einmal den Artikel 88203 ziehen.<br /><\/p><p class=\"MsoNormal\">&nbsp;<br /><\/p><p class=\"MsoNormal\">Könntest du veranlassen, dass die EDV die Aufträge dann so umwandelt, dass bei Bestellungen mit der o.g. EAN das Display 66924 genommen wird<br /><\/p><p class=\"MsoNormal\">und nicht der Einzelartikel 88203.<br /><\/p><p class=\"MsoNormal\">&nbsp;<br /><\/p><p class=\"MsoNormal\">Vielen Dank im Voraus!<br /><\/p><p class=\"MsoNormal\">&nbsp;<br /><\/p><p class=\"MsoNormal\">Gruß<br /><\/p><p class=\"MsoNormal\">&nbsp;<br /><\/p><p class=\"MsoNormal\">&nbsp;<br /><\/p><table style=\"width: 435pt\" width=\"0\" cellspacing=\"0\" cellpadding=\"0\" border=\"0\"><tbody><tr style=\"height: 78pt\"><td style=\"width: 91.5pt; padding: 0cm 0cm 0cm 0cm; height: 78pt\" width=\"122\"><p class=\"MsoNormal\"><span class=\"font\" style=\"font-family: &quot;Times New Roman&quot;, serif\"><span class=\"size\" style=\"font-size: 12pt\"><img alt=\"PAPSTAR Logo\" src=\"/inlineimages/WorkOrder/70840/image001.jpg\" style=\"width: 0.927in; height: 1.0416in\" width=\"0.927in\" height=\"1.0416in\" border=\"0\" /><\/span><\/span><br /><\/p><\/td><td style=\"width: 124.5pt; padding: 0cm 0cm 0cm 0cm; height: 78pt\" width=\"164\" valign=\"top\"><p class=\"MsoNormal\"><span class=\"font\" style=\"font-family: &quot;Times New Roman&quot;, serif\"><span class=\"size\" style=\"font-size: 7.5pt\">&nbsp;<\/span><span class=\"size\" style=\"font-size: 12pt\"><\/span><\/span><br /><\/p><table cellspacing=\"0\" cellpadding=\"0\" border=\"0\"><tbody><tr><td style=\"padding: 0cm 0cm 0cm 0cm\"><p class=\"MsoNormal\"><b><span class=\"colour\" style=\"color: black\"><span class=\"font\" style=\"font-family: &quot;Arial&quot;, sans-serif\"><span class=\"size\" style=\"font-size: 10pt\">PAPSTAR GmbH<\/span><\/span><\/span><\/b><span class=\"colour\" style=\"color: black\"><span class=\"font\" style=\"font-family: &quot;Times New Roman&quot;, serif\"><span class=\"size\" style=\"font-size: 10pt\"><\/span><\/span><\/span><br /><\/p><\/td><\/tr><tr><td style=\"padding: 0cm 0cm 0cm 0cm\"><p class=\"MsoNormal\"><span class=\"colour\" style=\"color: black\"><span class=\"font\" style=\"font-family: &quot;Arial&quot;, sans-serif\"><span class=\"size\" style=\"font-size: 10pt\">Daimlerstraße 4-8<\/span><\/span><span class=\"font\" style=\"font-family: &quot;Times New Roman&quot;, serif\"><span class=\"size\" style=\"font-size: 10pt\"><\/span><\/span><\/span><br /><\/p><\/td><\/tr><tr><td style=\"padding: 0cm 0cm 0cm 0cm\"><p class=\"MsoNormal\"><span class=\"colour\" style=\"color: black\"><span class=\"font\" style=\"font-family: &quot;Arial&quot;, sans-serif\"><span class=\"size\" style=\"font-size: 10pt\">53925 Kall<\/span><\/span><span class=\"font\" style=\"font-family: &quot;Times New Roman&quot;, serif\"><span class=\"size\" style=\"font-size: 10pt\"><\/span><\/span><\/span><br /><\/p><\/td><\/tr><\/tbody><\/table><\/td><td style=\"width: 221.25pt; padding: 0cm 0cm 0cm 0cm; height: 78pt\" rowspan=\"3\" width=\"294\"><p class=\"MsoNormal\"><a href=\"https://www.papstar-storyteller.de/\"><span class=\"colour\" style=\"color: blue\"><span class=\"font\" style=\"font-family: &quot;Times New Roman&quot;, serif\"><span class=\"size\" style=\"font-size: 12pt\"><img alt=\"PAPSTAR Campaign\" src=\"/inlineimages/WorkOrder/70840/image002.jpg\" style=\"width: 3.0208in; height: 2.8125in\" width=\"3.0208in\" height=\"2.8125in\" border=\"0\" /><\/span><\/span><\/span><\/a><span class=\"font\" style=\"font-family: &quot;Times New Roman&quot;, serif\"><span class=\"size\" style=\"font-size: 12pt\"><\/span><\/span><br /><\/p><\/td><\/tr><tr style=\"height: 46.5pt\"><td style=\"padding: 0cm 0cm 0cm 0cm; height: 46.5pt\" colspan=\"2\" valign=\"top\"><table cellpadding=\"0\" border=\"0\"><tbody><tr><td style=\"padding: 0.75pt 0.75pt 0.75pt 0.75pt\" valign=\"top\"><p class=\"MsoNormal\"><span class=\"colour\" style=\"color: black\"><span class=\"font\" style=\"font-family: &quot;Arial&quot;, sans-serif\"><span class=\"size\" style=\"font-size: 10pt\">i. A. Melanie Reger<\/span><\/span><span class=\"font\" style=\"font-family: &quot;Times New Roman&quot;, serif\"><span class=\"size\" style=\"font-size: 10pt\"><\/span><\/span><\/span><br /><\/p><\/td><td style=\"width: 0.3pt; padding: 0cm 0cm 0cm 0cm\" width=\"6\"><p class=\"MsoNormal\">&nbsp;<br /><\/p><\/td><\/tr><tr><td style=\"padding: 0.75pt 0.75pt 0.75pt 0.75pt\" rowspan=\"2\" valign=\"top\"><p class=\"MsoNormal\"><b><span class=\"colour\" style=\"color: black\"><span class=\"font\" style=\"font-family: &quot;Arial&quot;, sans-serif\"><span class=\"size\" style=\"font-size: 10pt\">National Sales<\/span><\/span><\/span><\/b><span class=\"colour\" style=\"color: black\"><span class=\"font\" style=\"font-family: &quot;Times New Roman&quot;, serif\"><span class=\"size\" style=\"font-size: 10pt\"><\/span><\/span><\/span><br /><\/p><\/td><td style=\"width: 0.3pt; padding: 0cm 0cm 0cm 0cm\" width=\"6\"><p class=\"MsoNormal\">&nbsp;<br /><\/p><\/td><\/tr><tr><td style=\"width: 0.3pt; padding: 0cm 0cm 0cm 0cm\" width=\"6\"><p class=\"MsoNormal\">&nbsp;<br /><\/p><\/td><\/tr><\/tbody><\/table><\/td><\/tr><tr style=\"height: 75pt\"><td style=\"padding: 0cm 0cm 0cm 0cm; height: 75pt\" colspan=\"2\" valign=\"bottom\"><table style=\"width: 210pt\" width=\"0\" cellspacing=\"0\" cellpadding=\"0\" border=\"0\"><tbody><tr><td style=\"width: 45pt; padding: 0cm 0cm 0cm 0cm\" width=\"60\"><p class=\"MsoNormal\"><span class=\"colour\" style=\"color: black\"><span class=\"font\" style=\"font-family: &quot;Arial&quot;, sans-serif\"><span class=\"size\" style=\"font-size: 10pt\">Phone:<\/span><\/span><\/span><br /><\/p><\/td><td style=\"padding: 0cm 0cm 0cm 0cm\"><p class=\"MsoNormal\"><span class=\"colour\" style=\"color: black\"><span class=\"font\" style=\"font-family: &quot;Arial&quot;, sans-serif\"><span class=\"size\" style=\"font-size: 10pt\">+49 244183151<\/span><\/span><\/span><br /><\/p><\/td><\/tr><tr><td style=\"padding: 0cm 0cm 0cm 0cm\"><p class=\"MsoNormal\"><span class=\"colour\" style=\"color: black\"><span class=\"font\" style=\"font-family: &quot;Arial&quot;, sans-serif\"><span class=\"size\" style=\"font-size: 10pt\">Fax:<\/span><\/span><\/span><br /><\/p><\/td><td style=\"padding: 0cm 0cm 0cm 0cm\"><p class=\"MsoNormal\"><span class=\"colour\" style=\"color: black\"><span class=\"font\" style=\"font-family: &quot;Arial&quot;, sans-serif\"><span class=\"size\" style=\"font-size: 10pt\">+49 2441835151<\/span><\/span><\/span><br /><\/p><\/td><\/tr><tr><td style=\"padding: 0cm 0cm 0cm 0cm\"><p class=\"MsoNormal\"><span class=\"colour\" style=\"color: black\"><span class=\"font\" style=\"font-family: &quot;Arial&quot;, sans-serif\"><span class=\"size\" style=\"font-size: 10pt\">Email:<\/span><\/span><\/span><br /><\/p><\/td><td style=\"padding: 0cm 0cm 0cm 0cm\"><p class=\"MsoNormal\"><span class=\"colour\" style=\"color: black\"><span class=\"font\" style=\"font-family: &quot;Arial&quot;, sans-serif\"><span class=\"size\" style=\"font-size: 10pt\"><a title=\"Click to send email to Reger, Melanie\"><span class=\"colour\" style=\"color: black\">Melhkl-e.Reger@pk-k-lar.de<\/span><\/a><\/span><\/span><\/span><br /><\/p><\/td><\/tr><tr><td style=\"padding: 0cm 0cm 0cm 0cm\"><p class=\"MsoNormal\"><span class=\"colour\" style=\"color: black\"><span class=\"font\" style=\"font-family: &quot;Arial&quot;, sans-serif\"><span class=\"size\" style=\"font-size: 10pt\">Web:<\/span><\/span><\/span><br /><\/p><\/td><td style=\"padding: 0cm 0cm 0cm 0cm\"><p class=\"MsoNormal\"><span class=\"colour\" style=\"color: black\"><span class=\"font\" style=\"font-family: &quot;Arial&quot;, sans-serif\"><span class=\"size\" style=\"font-size: 10pt\"><a href=\"http://www.papstar.com\"><span class=\"colour\" style=\"color: black\">www.papstar.com<\/span><\/a><\/span><\/span><\/span><br /><\/p><\/td><\/tr><\/tbody><\/table><\/td><\/tr><\/tbody><\/table><p class=\"MsoNormal\">&nbsp;<br /><\/p><\/div>","responded_time":{"display_value":"Jan 2, 2020 02:14 PM","value":"1577970892415"},"has_dependency":false,"closure_info":{"requester_ack_comments":null,"closure_code":null,"closure_comments":null,"requester_ack_resolution":true},"has_conversation":true,"callback_url":null,"is_service_request":false,"urgency":{"name":"Normal","id":"3"},"is_shared":false,"has_request_initiated_change":false,"request_template_task_ids":[],"department":{"site":{"id":"602"},"name":"Sales","id":"4209"},"is_reopened":true,"has_draft":false,"has_attachments":false,"has_linked_requests":false,"is_overdue":false,"technician":{"email_id":"Nhk-k.Hk-lig@pk-lk-tar.de","name":"Hk-kl-nig, Nik-klk","is_vipuser":false,"id":"78607","department":{"site":{"name":"Verwaltung Kall","id":602},"name":"IT","id":6902}},"has_request_caused_by_change":false,"has_problem":false,"due_by_time":{"display_value":"Jan 20, 2020 05:23 PM","value":"1579537382685"},"is_fcr":false,"has_project":false,"site":{"name":"Verwaltung Kall","id":"602"},"is_first_response_overdue":false,"completed_time":{"display_value":"Jan 10, 2020 09:06 AM","value":"1578643610287"},"category":{"name":"Anwendungsentwicklung","id":"3"}},"response_status":{"status_code":2000,"status":"success"}}


root@delxsearch:/docs/transform/in#

könnte sein, das vielleicht in einem feld etwas drinsteht das das stript verwirrt? da es sich ja teilweise um freitext handelt?

Ich habe das Script gerade mal über ein paar 100 laufen lassen, und nur bei den Wenigsten läuft das wir erwartet.

immer

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

File -> 34754
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  5384  100  5384    0     0  97890      0 --:--:-- --:--:-- --:--:-- 97890
curl: (3) URL using bad/illegal format or missing URL
Traceback (most recent call last):
  File "/docs/script/transform.py", line 41, in <module>
    result = {k: flattened[k] for k in keys}
  File "/docs/script/transform.py", line 41, in <dictcomp>
    result = {k: flattened[k] for k in keys}
KeyError: 'request.created_by.department.site.name'

ok, mit dem Neuen Script tritt der Fehler scheinbar nicht auf...

Jedoch geht die insert funktion nicht

1

 python3 /docs/script/transform2.py /docs/transform/out/$f.json -a '{"id": "$f"}'

ergebniss

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

File -> 71505
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 29140    0 29140    0     0   474k      0 --:--:-- --:--:-- --:--:--  474k
curl: (3) URL using bad/illegal format or missing URL
########## /docs/transform/out/71505.json ##########
Traceback (most recent call last):
  File "/docs/script/transform2.py", line 56, in <module>
    content = json.load(f)
  File "/usr/lib/python3.7/json/__init__.py", line 296, in load
    parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
  File "/usr/lib/python3.7/json/__init__.py", line 348, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.7/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.7/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

So wie du quotest, würde ein "$f" als Wert für "id" im String landen.

Die Fehlermeldung besagt dass die /docs/transform/out/$f.json keine gültige JSON-Datei ist - was zur Fehlermeldung von curl davor passen würde.

ok, das mit dem $f verstehe, aber keine Gültoge JSON datein?

ich mach das doch mit deinem Script.

1

 python3 /docs/script/transform2.py /docs/transform/in/$f.json > /docs/transform/out/$f.json

und das ergebniss davon will ich später mit weiteren inhalt füllen:

1

python3 /docs/script/transform2.py /docs/transform/out/$f.json -a '{"id": "$f"}'

AHHHH

es ist

1

########## /docs/transform/out/71472.json ##########

das wird von dem Script obben in die Datei geschrieben