How to separate predictions map to create id2label.json
Dear@John6666and other community membersSince using self-made mask didn’t work last time, said I need ground-truth image for the model to know what I mean by river area, how should I separate the “river” from the rest of tensors to make theid2label.json? Or did I do it the other way around by training the masked image?The examplesemantic_prediction[0]can spit out a tensor but I don’t think it means anything since matplotlib.pyplot.imshow(semantic_prediction[0]) can’t show anythingI’m currently making maps for each image, but so far the prediction are weirdly similar for different images, do their information differences negligible or should I do some normalization to make them seen?
2024-12-15T03:31:05Z
I’ve never tried segmentation before…I collected some links that might be useful if you’re trying to do segmentation with Hugging Face.How do I create a Image Segmentation Dataset🤗DatasetsI was able to solve that part by doing this: 
feature_extractor = SegformerFeatureExtractor(do_resize=False, do_normalize=False)

But I am getting another error 
[image] 
Not sure why this is cominghuggingface.coMask GenerationWe’re on a journey to advance and democratize artificial intelligence through open source and open science.huggingface.coFine-Tune a Semantic Segmentation Model with a Custom DatasetWe’re on a journey to advance and democratize artificial intelligence through open source and open science.
2024-12-15T03:37:54Z
Thanks I’ll check them out, sorry for keep asking you while you keep highlighting never done segmentation before, almost ran out of people to ask and I don’t know exactly what I’m trying to do
2024-12-15T03:43:47Z
No problem.It’s also a learning experience for me. However, if you want to ask someone who knows a lot about AI models, it may be quicker to use HF Discord.If you’re lucky, HF staff or someone knowledgeable will pass by the forum…
2024-12-15T03:52:16Z
The thing is tagging you has been the most dependable results I had so far even compared to discord forum,Read the forum, it’s exactly what I’m looking for, will try it first then ask again if I need some clarification; maybe there maybe here, thanks John.
2024-12-15T07:41:45Z
@John6666Okay so I get theseimage823×789 6.68 KBand this[{'score': None,
  'label': 'wall',
  'mask': <PIL.Image.Image image mode=L size=1080x1080>},
 {'score': None,
  'label': 'building',
  'mask': <PIL.Image.Image image mode=L size=1080x1080>},
 {'score': None,
  'label': 'sky',
  'mask': <PIL.Image.Image image mode=L size=1080x1080>},
 {'score': None,
  'label': 'tree',
  'mask': <PIL.Image.Image image mode=L size=1080x1080>},
 {'score': None,
  'label': 'mountain',
  'mask': <PIL.Image.Image image mode=L size=1080x1080>},
 {'score': None,
  'label': 'water',
  'mask': <PIL.Image.Image image mode=L size=1080x1080>},
 {'score': None,
  'label': 'fence',
  'mask': <PIL.Image.Image image mode=L size=1080x1080>},
 {'score': None,
  'label': 'rock',
  'mask': <PIL.Image.Image image mode=L size=1080x1080>},
 {'score': None,
  'label': 'signboard',
  'mask': <PIL.Image.Image image mode=L size=1080x1080>},
 {'score': None,
  'label': 'bridge',
  'mask': <PIL.Image.Image image mode=L size=1080x1080>},
 {'score': None,
  'label': 'boat',
  'mask': <PIL.Image.Image image mode=L size=1080x1080>},
 {'score': None,
  'label': 'pier',
  'mask': <PIL.Image.Image image mode=L size=1080x1080>}]Is this what I move to here?import datasets
import glob

IMAGES = glob.glob("D:\cropped_image_1080x1080\img_0")
SEG_MAPS = glob.glob("D:.\img_0.jpg")

dataset = datasets.Dataset.from_dict({
    "image": IMAGES,
    "label": SEG_MAPS
},
features=datasets.Features({
    "image":datasets.Image(),
    "label":datasets.Image()
                            })
)I still don’t get how I move the fifth member of thispipelinethat I got from herefrom transformers import pipeline

semantic_segmentation = pipeline("image-segmentation", "shi-labs/oneformer_ade20k_swin_large")to id2label, do I need to label it as implied inhere?
2024-12-15T07:51:11Z
How do this work for 8725 images that might have different scores, I still don’t understand the concept behind id2label and pretrained model…
2024-12-15T07:54:27Z
Oh, it seems to have worked out well. No matter which model you use, it’s better to refer to the author’s model card rather than the automatically generated upper part of the HF.For this model, it would look something like the link below.The format required will vary depending on the model, but as long as you make the dataset in a way that is easy for you to use, that should be fine.Uploading and downloading to HF’s dataset repo and organizing large amounts of data is the forte of the datasets library.However, there may be a recommended dataset format for HF Trainer.In the end, it’s the same thing that we need to write a few lines of Python before passing it to the model.I don’t think you need labels for use cases that don’t require labels. I don’t think you need labels for segmentation.huggingface.coshi-labs/oneformer_ade20k_swin_large · Hugging FaceWe’re on a journey to advance and democratize artificial intelligence through open source and open science.
2024-12-15T07:56:07Z
Then do I only need the mask for training? Every guide I found use id2label though, I understand that this is semantic segmentation so not only it is segmented but its also labelled as something
2024-12-15T07:57:12Z
Pipelines are convenient, but they perform complex processing internally, so they are like black boxes.It might be better to try doing it manually at first. The method on the model explanation page is the most general-purpose method.Then do I only need the mask for training?Yes.semantic segmentation so not only it is segmented but its also labelled as somethingIf that’s the case, you may need to label the data set…It’s difficult to create data that doesn’t exist. Only a large AI or a human being could do it.
2024-12-15T08:01:37Z
so I need to id2label for my mask, what should I do with each weight? Sorry I’m still confused and don’t know how to explain it tbh…I know I need these weights for some fine tuning but I don’t know where to store it as. Or if I train my mask I got from applying the model to my picture is it what I need?
2024-12-15T08:06:20Z
Even if you have to create your own data in the end, semantic segmentation itself is an existing task.I think there are already working datasets for semantic segmentation that you can use as samples. If you can imitate that structure, it will be the quickest.
2024-12-15T08:09:26Z
It seems like you need a mask that uses multiple colors or shades to semantic segmentation. Your mask is black and white, 0 or 1, so it should be usable for two types of classification. I think it should be usable for whether or not it’s a river.colab.research.google.comGoogle Colabhuggingface.cohuggingface/semantic-segmentation-test-sample · Datasets at Hugging FaceWe’re on a journey to advance and democratize artificial intelligence through open source and open science.huggingface.comoritzef/mapillary_vistas_semantic_edges_and_segmentation · Datasets at...We’re on a journey to advance and democratize artificial intelligence through open source and open science.by json file like this (or opposite?):{"0": "not river", "1": "river"}With HF, JSON files that have been carelessly left lying around are sometimes configuration files with fixed file names…
2024-12-15T20:09:29Z
This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.