Home | Community | Message Board


This site includes paid links. Please support our sponsors.


Welcome to the Shroomery Message Board! You are experiencing a small sample of what the site has to offer. Please login or register to post messages and view our exclusive members-only content. You'll gain access to additional forums, file attachments, board customizations, encrypted private messages, and much more!

Shop: Bridgetown Botanicals CBD Concentrates   Original Sensible Seeds Autoflowering Cannabis Seeds   Mushroom-Hut Mono Tub Substrate   PhytoExtractum Buy Bali Kratom Powder   Left Coast Kratom Buy Kratom Capsules   Kraken Kratom Buy Kratom Powder & Leaf

Jump to first unread post Pages: 1 | 2 | Next >  [ show all ]
InvisibleAndyHinton


Registered: 12/05/16
Posts: 434
BioTorrents.de, a platform to share biological sequence and medical imaging data * 1
    #26521916 - 03/07/20 09:29 AM (3 years, 11 months ago)

Hi guys,

I'm humbled to share with you a website that I've been working on for the last year, that I think fills an important niche in scientific data sharing.





BioTorrents.de is a semi-private torrent tracker where you can download genetic and protein sequences, medical images such as MRIs and X-rays, and the documentation and tools people wrote to work with the data. It uses the familiar Gazelle software so the metadata is really easy to search and the interface is really comfy, like What.cd for genomes instead of music.

The site also has a forum section that's heavily inspired by the old Shroomery vibe. There are places to discuss and learn about lab work, bioinformatics, journal articles, science news, and a marketplace for power users. It's like the advanced mycology board broken out into different sections.

I wrote the site because there are lots of small labs outside of universities who do cool work and find it difficult to collaborate. Often the data is hard to get at, for example, you must know a specific person to ask for access to a Google Drive. There's also something of a divide between wetlab and bioinformatics people.

BioTorrents.de is primarily for community biology labs, independent researchers, interested computer scientists, and open access data volunteers who don't have a big institution behind them. I believe that getting all these people together, and having the data right there to download and study, can galvanize hobby biology and make more people capable of more sophisticated experiments.

Did you know, for example, that in 2017 a group of researchers sequenced 10× more of the kratom genome than ever before, and released it anonymously on BitTorrent to stop big pharma companies from patenting it? That effort inspired me to try the idea again, and of course the Kratom Genome Project's releases are all readily available.

There was a site called BioTorrents that started in 2010, which I think was several years ahead of its time. My own site likewise won't last forever, so one of my priorities is to make it easy for others to restore the metadata, torrent files, and software. I want the site to belong to everyone who contributes to it.

Anyway, please find some screenshots below. If you're interested in the technical details, please read about the technology and review the source code.



Please remember that development is in progress, and that Easter is the official launch date. I'm looking at ways to integrate the site more fully with resources such as Sci-Hub, ORCiD author identification, and existing repositories like RefSeq and UniProt.


--------------------


Edited by AndyHinton (03/07/20 09:34 AM)


Extras: Filter Print Post Top
OfflineSolipsis
m̶a̶d̶ disappointed scientist
Male User Gallery


Registered: 12/28/09
Posts: 3,398
Loc: the Neitherlands Flag
Last seen: 6 months, 6 days
Re: BioTorrents.de, a platform to share biological sequence and medical imaging data [Re: AndyHinton]
    #26522786 - 03/07/20 07:48 PM (3 years, 11 months ago)

Great project, Andy!

<3 <3

I think a friend has a Ganoderma lucidum sequenced that could be shared


Extras: Filter Print Post Top
InvisibleAndyHinton


Registered: 12/05/16
Posts: 434
Re: BioTorrents.de, a platform to share biological sequence and medical imaging data [Re: Solipsis]
    #26522949 - 03/07/20 09:45 PM (3 years, 11 months ago)

Thanks for agreeing to join early, Solipsis, and for being patient as I flooded the site with hundreds of junk torrents testing new features. Now is a good time to use the development version for that. :thumbup:

I happily noticed a few people joined today, which is encouraging at this young stage. I also uploaded the enzymatic synthesis of psilocybin (JGI source) with all the gene sequences involved in psilocybin production.

Some DIYbio original data includes BosLab's 16S MiSeq reads from 6 soil samples, Counter Culture Labs's/BioCurious's 16S uBiome reads from kombucha, and the complete BioBrick parts repos.

With the collections feature, there's a place for my forthcoming fungi barcodes and any similar data Alan may wish to add. I also have some interesting X-rays on the way (longitudinal case study) and I want to get people to scan and upload their prenatal ultrasounds. Can we make that a thing? :crazy:


--------------------


Edited by AndyHinton (03/07/20 10:06 PM)


Extras: Filter Print Post Top
OfflinePTreeDish
Male


Registered: 04/22/18
Posts: 353
Last seen: 4 months, 2 days
Re: BioTorrents.de, a platform to share biological sequence and medical imaging data [Re: AndyHinton]
    #26527939 - 03/10/20 06:27 PM (3 years, 11 months ago)

Thank you for this site and nice work! The launch coincides nicely with my current foray into biosynthetic engineering.

FYI - the registration confirmation email went to spam in gmail. I marked as safe to help improve your spam score, but you might want to dig into any low-hanging fruit that could improve your email deliverability (spf record, etc).


Extras: Filter Print Post Top
InvisibleAndyHinton


Registered: 12/05/16
Posts: 434
Re: BioTorrents.de, a platform to share biological sequence and medical imaging data [Re: PTreeDish]
    #26528779 - 03/11/20 09:02 AM (3 years, 11 months ago)

Thanks. I'm aware of the spam problem and will definitely look at the DNS situation. I had gone through it before when Gmail started demanding IPv6 records, and will double check it now with a more careful eye. It could be that the cheap VPS host's IP range is untrusted by default. I'll also add notices to check your spam folder on all the pages that involve receiving email from the site.

Okay it's done. I made sure that the DNS config contained proper SPF and CAA records. While I was in there, I also updated the nameservers and enabled DNSSEC validation. I also started adding "please check your spam folder" messages to the relevant pages.


--------------------


Edited by AndyHinton (03/11/20 09:51 AM)


Extras: Filter Print Post Top
OfflinePTreeDish
Male


Registered: 04/22/18
Posts: 353
Last seen: 4 months, 2 days
Re: BioTorrents.de, a platform to share biological sequence and medical imaging data [Re: AndyHinton]
    #26528932 - 03/11/20 10:46 AM (3 years, 11 months ago)

Roger - might take a little bit for the changes to propagate but it's usually pretty fast these days. To circumvent the VPS IP issue, I typically use AWS SES or similar for these kinds of transnational emails. Cost is negligible.

I checked the IPs for you and they don't appear to be on any blacklists. You might still want to add a DMARC txt record to DNS.

I'll see if there's a way I can trigger an email later today and see if it goes to Spam. Last night, it looks like a few notifications still went to spam but that was before you made these changes.


Extras: Filter Print Post Top
InvisibleAndyHinton


Registered: 12/05/16
Posts: 434
Re: BioTorrents.de, a platform to share biological sequence and medical imaging data [Re: PTreeDish]
    #26529200 - 03/11/20 01:48 PM (3 years, 11 months ago)

DKIM signing has been on my todo list for all my email servers. I'll make it a priority: after getting IRC announces set up and before starting on several larger plans such as reworking the JSON API.

DMARC can probably happen much sooner as it doesn't need me to proxy anything to the SMTP server. Then I can update the DMARC record later. Thanks again for helping test this. I've been using password reset requests for my own testing, which expire in 1 hour.


--------------------


Extras: Filter Print Post Top
OfflineeLeSDenes
Mycelium Expander
Male User Gallery


Folding@home Statistics
Registered: 05/25/11
Posts: 955
Last seen: 2 years, 6 days
Re: BioTorrents.de, a platform to share biological sequence and medical imaging data [Re: AndyHinton]
    #26529261 - 03/11/20 02:30 PM (3 years, 11 months ago)

well done man nice site, I even registered. I know quite a bit of computer science and AI but have no idea about genes and sequencing, might be a nice hobby to pick up.


Extras: Filter Print Post Top
OfflinePTreeDish
Male


Registered: 04/22/18
Posts: 353
Last seen: 4 months, 2 days
Re: BioTorrents.de, a platform to share biological sequence and medical imaging data [Re: AndyHinton]
    #26529264 - 03/11/20 02:34 PM (3 years, 11 months ago)

Quote:

AndyHinton said:
DKIM signing has been on my todo list for all my email servers. I'll make it a priority: after getting IRC announces set up and before starting on several larger plans such as reworking the JSON API.

DMARC can probably happen much sooner as it doesn't need me to proxy anything to the SMTP server. Then I can update the DMARC record later. Thanks again for helping test this. I've been using password reset requests for my own testing, which expire in 1 hour.




This should be all you need, replacing the email address with where you want to receive DMARC reports and failures.

Code:
Type: TXT
Host/Name: _dmarc.biotorrents.de
Value: v=DMARC1; p=none; rua=mailto:foo@example.com; ruf=mailto:foo@example.com



I see you're using php. What's the rest of your stack as it pertains to the api? From the page names, doesn't seem obvious that you're using a framework like Laravel or Zend. (If you so care to share publicly)

And I'm happy to test future stuff - just give me a hollar.


Extras: Filter Print Post Top
InvisibleAndyHinton


Registered: 12/05/16
Posts: 434
Re: BioTorrents.de, a platform to share biological sequence and medical imaging data [Re: PTreeDish]
    #26530506 - 03/12/20 08:45 AM (3 years, 11 months ago)

Thanks, eLeSDenesS. There are some cool things happening in the neighborhood of biology and computer science. This is an effort to help bring the two fields closer together, especially in the DIYbio scene. Sharing data, even among members of the same lab, is often more difficult than it should be.

For example, it took me weeks to get access to the BosLab's soil metagenomic data, even though I'm friends with the people who produced it and could directly reach out to the specific person who set up the Google Drive. Last year I watched about five interested bioinformatics people come and go for lack of easy data access.

Quote:

PTreeDish said:
I see you're using php. What's the rest of your stack as it pertains to the api? From the page names, doesn't seem obvious that you're using a framework like Laravel or Zend. (If you so care to share publicly)

And I'm happy to test future stuff - just give me a hollar.



Thanks. I applied DMARC to all my DNS zones. The codebase is a custom framework called Gazelle that What.cd developed in house. BioTorrents.de's source code is based on Oppaitime's security-hardened fork of Gazelle. I took the liberty to set up a repo at Oppaitime's Git.

That's where a lot of tracker development happens these days, though there are sites using GitHub such as Orpheus and AnimeBytes. Unfortunately many of the best torrent sites don't have source code available.


--------------------


Extras: Filter Print Post Top
OfflinePTreeDish
Male


Registered: 04/22/18
Posts: 353
Last seen: 4 months, 2 days
Re: BioTorrents.de, a platform to share biological sequence and medical imaging data [Re: AndyHinton]
    #26531674 - 03/12/20 08:11 PM (3 years, 10 months ago)

Quote:

Thanks. I applied DMARC to all my DNS zones. The codebase is a custom framework called Gazelle that What.cd developed in house. BioTorrents.de's source code is based on Oppaitime's security-hardened fork of Gazelle. I took the liberty to set up a repo at Oppaitime's Git.

That's where a lot of tracker development happens these days, though there are sites using GitHub such as Orpheus and AnimeBytes. Unfortunately many of the best torrent sites don't have source code available.




Cool - I'll keep an eye out on my notifications and see if they start coming through. 🎲

Quote:


Thanks. I applied DMARC to all my DNS zones. The codebase is a custom framework called Gazelle that What.cd developed in house. BioTorrents.de's source code is based on Oppaitime's security-hardened fork of Gazelle. I took the liberty to set up a repo at Oppaitime's Git.

That's where a lot of tracker development happens these days, though there are sites using GitHub such as Orpheus and AnimeBytes. Unfortunately many of the best torrent sites don't have source code available.




Cool, never heard of Oppaitime until now but upon research, I realize it's not my -ahem- preferred genre -ahem- for private torrents, so we wouldn't have crossed paths. :tongue:

What's the advantage of having these private torrents vs. just sharing it with one of the existing big trackers that are aggregated say on the pirate bay? Especially for science stuff. Just curious.


Extras: Filter Print Post Top
InvisibleAndyHinton


Registered: 12/05/16
Posts: 434
Re: BioTorrents.de, a platform to share biological sequence and medical imaging data [Re: PTreeDish]
    #26535208 - 03/14/20 05:04 PM (3 years, 10 months ago)

PTreeDish, you'll be glad to hear that full DKIM/DMARC/SPF compliance is in effect, as are A/AAAA and rDNS entries. The rest is out of my control, as the domain name is young and the host is cheap and spammy.

The benefits of the private tracker model are great organization and accountability for your downloads. The site can search by any metadata and it enforces ratio requirements to ensure that real seeds are available.

Compared to FTP/HTTP web seeds, which tend to stall out and/or crash the client. But limited support for web seeds is planned and the DB currently stores mirrors.

Oppaitime has the best torrent tracker software available today.


--------------------


Extras: Filter Print Post Top
OfflinePTreeDish
Male


Registered: 04/22/18
Posts: 353
Last seen: 4 months, 2 days
Re: BioTorrents.de, a platform to share biological sequence and medical imaging data [Re: AndyHinton]
    #26537619 - 03/15/20 10:37 PM (3 years, 10 months ago)

:dancer: Lookin' good! Appreciate the insight on private trackers. :thumbup:


Extras: Filter Print Post Top
InvisibleAndyHinton


Registered: 12/05/16
Posts: 434
Re: BioTorrents.de, a platform to share biological sequence and medical imaging data [Re: PTreeDish]
    #26605116 - 04/16/20 08:07 AM (3 years, 9 months ago)

A quick bump to let you guys know that BioTorrents.de officially launched on Easter Monday. I'll start to announce it in DIYbio circles, which should bring in some quirky scientists. :peace:


--------------------


Extras: Filter Print Post Top
OfflinePTreeDish
Male


Registered: 04/22/18
Posts: 353
Last seen: 4 months, 2 days
Re: BioTorrents.de, a platform to share biological sequence and medical imaging data [Re: AndyHinton]
    #26606319 - 04/16/20 05:45 PM (3 years, 9 months ago)

Hey man - awesome, congrats. Not a lot of folks can claim they saw a project through to launch. Well done sir!

Also, I noticed this in your recent update:

Quote:

Relevant to BioTorrents.de: I bought a microscope because it's fun to look at tiny things! The microscope is fairly easy to modify and lets me make beautiful polarized light, darkfield, and (coming soon) fluorescence images. The first proper dataset I'll use it for is a collection of filamentous fungi before and after storage in a novel cryogenic medium.




What scope did you end up getting? Can you share more about what you plan to do, including more details on the cry aspect? I'd love to follow along and do something similar.


Extras: Filter Print Post Top
InvisibleAndyHinton


Registered: 12/05/16
Posts: 434
Re: BioTorrents.de, a platform to share biological sequence and medical imaging data [Re: PTreeDish] * 2
    #26606396 - 04/16/20 06:18 PM (3 years, 9 months ago)

For sure. It's a Nikon Labophot with 4 plan objectives, refurbished by a pro. The mods and dSLR attachment are inspired by this guide.

It's possible to do polarized light by placing two disks of polarizing film at different angles relative to each other. Darkfield is also easy: put a coin over the light source. I also want to look into fluorescence with a blue LED array and amber film.

The next step up in mods would be confocal, which uses pinhole light and a beam splitter to decrease the z-axis depth of field. I want to see if it's possible to capture confocal images directly with a camera on astrophotography settings (really long exposures), because making a laser to USB scanner seems difficult to say the least.

The cryo details are at diybio/structured-water. I'm expecting the hyphae to compress by about 50% after some months in 20% DMSO : 10% glycerol : 0.3% sea salt/malt extract.


--------------------


Edited by AndyHinton (04/16/20 06:26 PM)


Extras: Filter Print Post Top
OfflinePTreeDish
Male


Registered: 04/22/18
Posts: 353
Last seen: 4 months, 2 days
Re: BioTorrents.de, a platform to share biological sequence and medical imaging data [Re: AndyHinton]
    #26608308 - 04/17/20 02:39 PM (3 years, 9 months ago)

Really cool. Thanks for sharing!


Extras: Filter Print Post Top
Offlinepoisoned
untitled
I'm a teapot User Gallery


Registered: 04/17/13
Posts: 1,738
Loc: Yurop
Last seen: 1 year, 3 months
Re: BioTorrents.de, a platform to share biological sequence and medical imaging data [Re: PTreeDish]
    #26614605 - 04/20/20 08:14 AM (3 years, 9 months ago)

Just checked out the site, looks awesome but most stuff there seems a bit above my skill set or knowledge


--------------------
How I do glass dishes


Extras: Filter Print Post Top
InvisibleAndyHinton


Registered: 12/05/16
Posts: 434
Re: BioTorrents.de, a platform to share biological sequence and medical imaging data [Re: poisoned]
    #27048013 - 11/19/20 08:13 PM (3 years, 2 months ago)

A quick bump to announce the first major milestone: 100 GB data tracked.

Things are coming along nicely and I've been doing a lot of development work lately. There's a new default theme (advanced search screenshot below), lots of bugs fixed, lots of junk features removed, major server upgrades including the MariaDB database, and a couple new features such as onsite password generator and OpenDyslexic font support.

The biggest change since I last checked in: the site categories are totally revamped to support a large variety of biological data. The new organization comes from Catalyzing Inquiry at the Interface of Computing and Biology, Chapter 3. Sequences and images are the tip of the iceberg. There's a place for data such as regulatory networks, protein structural data, microfluidics and time-dependent biochemistry data, models such as the data produced by Folding@home, documentation and disk images, etc.

This is all gearing up for the real stuff. Currently I'm working through the scripting/automation interfaces like RSS feeds, JSON API, and IRC server. I've also got some upload form niceties planned like AJAX form autofill by DOI number, format and compression detection, and protocol-agnostic mirror support for at least HTTP/FTP, Dat, and IPFS.

That's the current direction of the development, and winter is the season to make it happen. I'll check in again around Easter, when the site officially turns one year old. Thanks for all your bug reports and suggestions, guys.



--------------------


Edited by AndyHinton (11/19/20 08:23 PM)


Extras: Filter Print Post Top
OfflinePTreeDish
Male


Registered: 04/22/18
Posts: 353
Last seen: 4 months, 2 days
Re: BioTorrents.de, a platform to share biological sequence and medical imaging data [Re: AndyHinton]
    #27048688 - 11/20/20 10:12 AM (3 years, 2 months ago)

Nice work! Congrats on the growth.


Extras: Filter Print Post Top
Jump to top Pages: 1 | 2 | Next >  [ show all ]

Shop: Bridgetown Botanicals CBD Concentrates   Original Sensible Seeds Autoflowering Cannabis Seeds   Mushroom-Hut Mono Tub Substrate   PhytoExtractum Buy Bali Kratom Powder   Left Coast Kratom Buy Kratom Capsules   Kraken Kratom Buy Kratom Powder & Leaf


Similar ThreadsPosterViewsRepliesLast post
* Growing in Liquid Medium. Using knowledge from Reactor Des
( 1 2 3 all )
Randoscious_McCray 15,674 51 03/19/05 12:56 AM
by Randoscious_McCray
* Increasing potency through species sequencing... LynxRufus 2,650 11 11/20/05 12:10 AM
by Zen Peddler
* Strain sequencing on SMS Jeremy_Davis 4,726 0 06/25/05 12:42 PM
by Jeremy_Davis
* Biological Efficiency
( 1 2 all )
Alkaloids 4,731 27 03/17/05 06:34 PM
by Joshua
* biology mrsoul 872 2 06/14/02 07:13 AM
by CLuB99
* sharing my recent disasters blackout 839 2 08/21/02 09:01 PM
by blackout
* anyone want to share their..... FuNgUsMaN420 2,067 18 11/03/02 06:55 PM
by hsalf
* Terrarium specifics, please share =) amyloid 2,503 13 06/23/03 03:59 PM
by PsilocyberBabe

Extra information
You cannot start new topics / You cannot reply to topics
HTML is disabled / BBCode is enabled
Moderator: RogerRabbit, Pastywhyte, bodhisatta
1,424 topic views. 0 members, 6 guests and 2 web crawlers are browsing this forum.
[ Show Images Only | Sort by Score | Print Topic ]
Search this thread:

Copyright 1997-2024 Mind Media. Some rights reserved.

Generated in 0.025 seconds spending 0.006 seconds on 14 queries.