Python Programming & other inferior programming languages thread

Discussion in 'The Mainboard' started by Walt Disney, Jul 15, 2015.

  1. Walt Disney

    Walt Disney #Dawgzy
    Donor TMB OG
    Georgia BulldogsManchester City

    Team vscode
     
    RegimentML likes this.
  2. Where Eagles Dare

    Where Eagles Dare The Specialist Show On Earth
    Donor
    Auburn TigersAtlanta BravesWashington Football TeamAtlanta United

    I've been using it a lot for python and sql queries.

    For python idk where the fuck my home directory is and can't ever find files I write that aren't explicitly defined (this is probably something stupid on me) . Also like it for sql, just wish it had a better connection into redshift. I just hate sql workbench J.

    Like the terminal and other connections. AWS connection tools are pretty dope
     
  3. Pile Driving Miss Daisy

    Pile Driving Miss Daisy It angries up the blood
    Donor
    Texas LonghornsAtlanta BravesAtlanta HawksAtlanta FalconsAtlanta UnitedGeorgia Southern Eagles

    If you don't mind spending a little bit of money, TablePlus is one of the best GUI tools that will connect to Redshift if you aren't on Linux. It works with basically every modern SQL DB.

    https://tableplus.com/
     
    Doc Louis and Henry Blake like this.
  4. Where Eagles Dare

    Where Eagles Dare The Specialist Show On Earth
    Donor
    Auburn TigersAtlanta BravesWashington Football TeamAtlanta United

    That actually looks pretty nice.

    If I had to stay on this project for any longer than this year I'd might think about it
     
  5. pratyk

    pratyk Arsenal FC, Rutgers Scarlet Knights
    Rutgers Scarlet KnightsArsenal

    Pycharm professional version has a nice built in SQL IDE. They do sell it separately as Datagrip. Worth checking out imo
     
    Guns likes this.
  6. kennypowers

    kennypowers Big shit like a dinosaur did it
    Donor TMB OG
    UCF KnightsAtlanta BravesJacksonville Jaguars

    Linux or windows?
     
  7. kennypowers

    kennypowers Big shit like a dinosaur did it
    Donor TMB OG
    UCF KnightsAtlanta BravesJacksonville Jaguars

    import os
    print(os.path.expanduser('~'))

    If you want to change it then you’ll need to modify your environment variables.
     
    Where Eagles Dare likes this.
  8. Where Eagles Dare

    Where Eagles Dare The Specialist Show On Earth
    Donor
    Auburn TigersAtlanta BravesWashington Football TeamAtlanta United

    Windows
     
  9. Henry Blake

    Henry Blake No Springsteen is leaving this house!
    Donor

    Anyone familiar with MariaDB? I have used MySQL (w PHP) in the past and may use Maria at some point. Should I expect to be able to use essentially the same PHP code to connect/select/delete/etc.? Pretty basic stuff for what I need. Not many involved queries or operations.
     
  10. —

    Well-Known Member
    Donor

    Trying to create a website that scrapes information and sends out text notifications based on values from various websites. Seems simple but I know it isn't. Five years ago I worked with a developer on a scraping tool that worked pretty well for the data we were looking for, however this was in 2016 and I found during the course of working with him that he was a maga chud/proudboy adjacent wacko, so I'm not going to call him again.

    Any resources for finding a moderately priced back end developer? Was checking out fiverr and man there really aren't great back end options. Lots of people with 2 reviews total. Seems fiverr developers specialize in making business websites for boomers, not actually writing code. Anyone in here looking for a little side work? TIA GBR
     
  11. Where Eagles Dare

    Where Eagles Dare The Specialist Show On Earth
    Donor
    Auburn TigersAtlanta BravesWashington Football TeamAtlanta United

    Upwork might be an option
     
  12. leroi

    leroi -
    Donor
    South Carolina GamecocksGrateful Dead

    lately i've been learning Docker Compose + Docker and i love it

    also learning Django and Postgres. Don't love them quite as much but we're getting there
     
  13. kentucky_dawg

    kentucky_dawg Fan of: Georgia
    Donor
    Georgia Bulldogs

    Now move onto Kubernetes, and you’ll hate it!

    I kid, Docker is good stuff and makes local development on a team really seamless.
     
    leroi likes this.
  14. Pile Driving Miss Daisy

    Pile Driving Miss Daisy It angries up the blood
    Donor
    Texas LonghornsAtlanta BravesAtlanta HawksAtlanta FalconsAtlanta UnitedGeorgia Southern Eagles

    Kubernetes orchestration is one of those tools that has been way too commonly thought as required. It's needed if you're orchestrating lots of microservices that serves thousands of clients that are disparate enough where a devops team needs to handle it, but after using it for my external API service it was much simpler to just do you regular deployment of a build to an EC2 instance that may or may not be clustered by simpler means.
     
    leroi likes this.
  15. BellottiBold

    Donor
    Oregon Ducks

    somehow mamp improved to the point that we switched back to it from Docker for local dev purposes... but we're a tiny operation so our choices are often weird
     
    leroi likes this.
  16. pratyk

    pratyk Arsenal FC, Rutgers Scarlet Knights
    Rutgers Scarlet KnightsArsenal

    IMO, Kubernetes is overkill for most projects. It's a complex solution for complex problems and more often than not, just not handled right so comes with a lot of baggage.
     
  17. kentucky_dawg

    kentucky_dawg Fan of: Georgia
    Donor
    Georgia Bulldogs

    It was a joke guyz
     
  18. pratyk

    pratyk Arsenal FC, Rutgers Scarlet Knights
    Rutgers Scarlet KnightsArsenal

    I hear you. Just didn't want anyone to dive into it headfirst and get a contusion setting up a cluster to serve a blog or something ;)
     
  19. BellottiBold

    Donor
    Oregon Ducks

    Someone help me with this annoying SQL matter...

    I'm trying to come up with a simple way to fix a table that we inadvertently munged up with weird duplicates, but I only care about said duplicates given a more specific criteria...

    Example:
    Code:
    id|A|B
    1 | orange | else
    2 | pear | else
    3 | lemon | thing
    4 | lemon | thing
    5 | apple | whatever
    6 | lemon | blah
    7 | orange | again
    
    Column B duplicates with a column A that has more than one distinct value are the rows that concern me.
    In other words, I care about rows 1, 2 because "else" appears twice and corresponds to more than one unique B value...
    1 - want
    2 - want
    3 - dgaf (same single A value)
    4 - dgaf (same single A value)
    5 - dgaf (totally unique)
    6 - dgaf (not a duplicate B value)
    7 - dgaf (not a duplicate B value)

    TIA <3 <3
     
  20. pratyk

    pratyk Arsenal FC, Rutgers Scarlet Knights
    Rutgers Scarlet KnightsArsenal

    Try a selfjoin with b=b and a<>a?
    Self Join is the way to go.. Here's a quick example.

    https://www.db-fiddle.com/f/3Hsmvy9oefKnXjG19sdUsc/0
     
    BellottiBold likes this.
  21. Ralph

    Ralph Well-Known Member
    Donor

    Nm, answered.
     
    #1721 Ralph, Jan 30, 2021
    Last edited: Jan 30, 2021
  22. Walt Disney

    Walt Disney #Dawgzy
    Donor TMB OG
    Georgia BulldogsManchester City

    Not sure I understand the goal but row number partition by is a good way to remove duplicates
     
  23. BellottiBold

    Donor
    Oregon Ducks

    pratyk and Walt Disney like this.
  24. pratyk

    pratyk Arsenal FC, Rutgers Scarlet Knights
    Rutgers Scarlet KnightsArsenal

    I am sure there are more elegant ways to handle this but sometimes it's better to start with the simplest and iterate using Explain Plan, if you're worried about the performance hit on full table scans - maybe look at adding some indices to the table to make it faster.
     
  25. BellottiBold

    Donor
    Oregon Ducks

    I'm not that worried about the performance in this case :)
    (We have plenty of leeway and the tables aren't that large to begin with)
     
  26. leroi

    leroi -
    Donor
    South Carolina GamecocksGrateful Dead

    I did some research into Kubernetes before I got started

    it seems like it'd be important if you were trying to scale something really huge

    but i am rolling the dice that it's overkill for my needs, betting that my little heroku stack will be enough

    but goddamn i'm happy with django. i know it's old, but it fits my approach to development perfectly. i should've started using this years ago.
     
  27. Pile Driving Miss Daisy

    Pile Driving Miss Daisy It angries up the blood
    Donor
    Texas LonghornsAtlanta BravesAtlanta HawksAtlanta FalconsAtlanta UnitedGeorgia Southern Eagles

    Kafka seems like something I've seen people reach for lately where you really need a lot of evidence and data to support using it. We're still using Angular 1.4 which is ancient in the frontend JS world, but I'm only part of a team with one other developer and my boss who has some scripting, DB, and a lot of networking knowledge and we don't have the time to just learn and rewrite our codebase with React since the number of clients who use our app are still too small. It would likely be a lot more easy to maintain in the future and easier to hire people as well as likely being faster to load overall, but ain't nobody got time for that when it's not 100% necessary.
     
    leroi likes this.
  28. leroi

    leroi -
    Donor
    South Carolina GamecocksGrateful Dead

    i think if newer web technologies actually help you solve a business problem, like they actually work a lot better for the user, then that's one thing. That's how my mapping stuff is, the newer software is just amazing and the old is shit and the user experience is on a totally different level.

    But for most other stuff? Yeah. Old tech works fine as long as its updated security-wise. When I describe things to my non-technical partners I use the analogy of a 1985 Toyota 4x4 with a carbureted engine and a solid front axle.
     
  29. NDfanPSUgrad

    NDfanPSUgrad Well-Known Member
    Penn State Nittany LionsNotre Dame Fighting Irish

    absolute shot in the dark here but I’ll give it a try. Just got word that our lead developer that is a stud is leaving my small team. We have a healthcare platform around predictive analytics that needs a high quality developer with machine learning and QA skills. Where is the best place to find this person? Happy to privately share more if interested. I’m not the technical guy but I can get answers quick. Position could eventually be based in Ann Arbor but this guy was 100% remote for years.
     
  30. Where Eagles Dare

    Where Eagles Dare The Specialist Show On Earth
    Donor
    Auburn TigersAtlanta BravesWashington Football TeamAtlanta United

    Probably through a recruiter if you don't have time to scour linkedin and message tons of ppl trying to find someone. Job listings usually suck.
     
    NDfanPSUgrad likes this.
  31. leroi

    leroi -
    Donor
    South Carolina GamecocksGrateful Dead

    You could post the listing on ycombinator, but keep in mind to learn the etiquitte of when/where to post the listings, and what sort of info you're expected to provide. They have a monthly Who's Hiring thread, this is the most recent :
    https://news.ycombinator.com/item?id=25989764

    I think Stack Exchange is supposed to be a good place to find developers

    Linkedin is ok i guess

    Recruiters, i mean that's an option, but i think a lot of people won't deal with recruiters.

    I guess you could use Dice? lol. if that still exists
     
    NDfanPSUgrad likes this.
  32. Where Eagles Dare

    Where Eagles Dare The Specialist Show On Earth
    Donor
    Auburn TigersAtlanta BravesWashington Football TeamAtlanta United

    Any ad you place you're going to get 1000 of responses, 99% will be unqualified and most will be offshore resources. Finding candidates for jobs is fucking brutal. You have to weed through copious amounts of shit to find someone good.
     
    Guns and NDfanPSUgrad like this.
  33. pratyk

    pratyk Arsenal FC, Rutgers Scarlet Knights
    Rutgers Scarlet KnightsArsenal

    Look for remote friendly developer job boards. YC/Hacker News always has some listed. The hiring thread is monthly and will be active next week
     
    NDfanPSUgrad and leroi like this.
  34. leroi

    leroi -
    Donor
    South Carolina GamecocksGrateful Dead

    Yeah, work involves work. That how it be

    also (true or not) healthcare tech has a reputation of being very bureaucratic and not-fun
     
    #1734 leroi, Feb 25, 2021
    Last edited: Feb 25, 2021
    NDfanPSUgrad likes this.
  35. VASooner

    VASooner Well-Known Member
    Donor
    Washington NationalsOklahoma City ThunderDallas CowboysOklahoma SoonersLiverpool

    Anyone good with Splunk? I need some help with a little project and I suck at it.
     
  36. pratyk

    pratyk Arsenal FC, Rutgers Scarlet Knights
    Rutgers Scarlet KnightsArsenal

    Writing queries or solution architecture itself?
     
  37. VASooner

    VASooner Well-Known Member
    Donor
    Washington NationalsOklahoma City ThunderDallas CowboysOklahoma SoonersLiverpool

    Queries. I have an access log and I can't for the life of me figure out when the "bad actor" gained access to a WordPress site. I see what was changed, etc, but not the initial login.
     
  38. Where Eagles Dare

    Where Eagles Dare The Specialist Show On Earth
    Donor
    Auburn TigersAtlanta BravesWashington Football TeamAtlanta United

    Anyone send emails from a MSOffice account with 2FA? My company just switched over to it and it broke all my scripts to send automated emails b/c it can't authenticate. :tebow:
     
  39. pratyk

    pratyk Arsenal FC, Rutgers Scarlet Knights
    Rutgers Scarlet KnightsArsenal

    Ahh. It's been a while since I looked at Splunk querying.

    Are you looking to identify client IP addresses / files /usernames? IIRC, splunk would auto-analyse logs and show client IP patterns / filenames under the "interesting fields" section and I would use a custom function to call an API (ipinfo.io) to grab more information about the client ip or segment file access by number of transfers / data consumption, etc.

    Edit: Just looked at my bookmarks and I used this site to find interesting queries that may have suited the task i was doing.

    https://gosplunk.com/
     
  40. Wee Bey

    Wee Bey racially ambiguous
    Donor

    Anyone mess around with Solidity? It's used on the Ethereum blockchain to create smart contracts.
     
  41. COVIDiskilingme79

    COVIDiskilingme79 Well-Known Member
    Donor
    Florida GatorsChicago Cubs

    I have a question for all you smart people in this thread. I have a Google sheets doc that I've used to track all of my personal flight data over the last three years. I keep track of origin airport, destination airport, airline flown on, duration of flight, and miles flown etc. I'm around 85 flights in on this sheet and it's starting to get bloated.

    I'm trying to think of the best way to streamline this and have decent querying abilities. Like if I wanted to show my 10 longest flights or all flights to/from an airport

    My first instinct is to go with SQL but I'm not sure if there's a simpler way to do this in P
     
  42. kennypowers

    kennypowers Big shit like a dinosaur did it
    Donor TMB OG
    UCF KnightsAtlanta BravesJacksonville Jaguars

    I’m almost always in favor of automating through programming…but in this case unless you have thousands of entries, it seems like having a Google sheet with the sort/filter option is easy and effective :idk:
     
    DollarBillHokie and Henry Blake like this.
  43. Henry Blake

    Henry Blake No Springsteen is leaving this house!
    Donor

    Yep. Sort/filter and set up a couple macros w buttons to do all the sorts you want.
     
    Doc Louis likes this.
  44. Doc Louis

    Doc Louis Well-Known Member
    Donor

    That's a lot of flights for 2018 and 2019.
     
  45. leroi

    leroi -
    Donor
    South Carolina GamecocksGrateful Dead

    This is mostly a question of how you're going to get data in, IMO. what you're really looking for is a simple and easy front end for data entry. What database you use doesn't really matter.

    Spreadsheets like Google Docs have a huge leg up on databases in terms of the ease of data entry
     
    Fusiontegra likes this.
  46. COVIDiskilingme79

    COVIDiskilingme79 Well-Known Member
    Donor
    Florida GatorsChicago Cubs

    Thanks. I have to rebuild my sheet from scratch because I outgrew it's original use case. Just thought I could streamline some things and make it a little more powerful.

    That's a good thought too. I was thinking I could build a shitty app to go with it for input but I'd be starting all of this from level 0 and just doing it in my free time.

    54 flights in 2019, 9 in 2020, and 18 so far in 2021. So I guess 81 total.
     
    Doc Louis likes this.
  47. Fusiontegra

    Fusiontegra My life is dope and I do dope shit.#SparedByThanos
    Donor TMB OG
    Miami MarlinsJacksonville JaguarsSporting Kansas CityAvengersMiami HeatSneakersWu-tangBook Club

    Because I function in the space, I know that a number of low code platforms would be perfect for exactly this. Whipping up something to do spreadsheet import would take like a day
     
  48. kennypowers

    kennypowers Big shit like a dinosaur did it
    Donor TMB OG
    UCF KnightsAtlanta BravesJacksonville Jaguars

    What would be really dope is if you scripted up a way to automate the entry based on something like a confirmation email or digital boarding pass.

    You could even trigger it to look up the flight info in FlightAware or something so you could add fields like how long you taxi’d or delays etc.
     
  49. Fusiontegra

    Fusiontegra My life is dope and I do dope shit.#SparedByThanos
    Donor TMB OG
    Miami MarlinsJacksonville JaguarsSporting Kansas CityAvengersMiami HeatSneakersWu-tangBook Club

    Email seems most likely there - especially if you fly the same airline and know what sort of info you’re looking in their automated emails. Using the digital boarding pass starts to involve figuring out said airline’s use of an api and for a side project, that seems overly involved
     
    BellottiBold and Doc Louis like this.
  50. kennypowers

    kennypowers Big shit like a dinosaur did it
    Donor TMB OG
    UCF KnightsAtlanta BravesJacksonville Jaguars

    Yeah pythonically it would definitely be easy to look through a plain text email for certain words.
     
    Fusiontegra likes this.